This document was developed in response to the W3C RDF Validation Workshop. The contributions are not associated with any W3C Working Group or any Recommendations-track work.

Namespace Prefixes

Most RDF languages have adopted the SPARQL conventions of BASE and PREFIX declarations. The behavior of these is detailed Turtle section 2.4 IRIs but briefly described here.

• Per RFC3986 section 5.1. Establishing a Base URI, web documents acquire a base URI from their access context. An HTTP GET on a resource like http://www.w3.org/2013/ShEx/test/Issue-opt-group/schema.shex will establish a BASE URI, which will be used to resolve relative URLs like <IssueShape>, unless the payload itself sets the BASE URI.
• The above example schema.shex includes a BASE directive which overrides the one established by the HTTP protocol.

The PREFIX directive provides associates a short string with a long URI called a namespace. These are used when writing URIs with the form prefix:localName. The URI denoted by this is the concatonation of namespace associated with the prefix, and the part to the right of the first ':' ("localName", in the above example).

Our example uses BASE and PREFIX directives:

BASE <http://base.example/#>
PREFIX ex: <http://ex.example/#>
PREFIX foaf: <http://foaf.example/#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

Start Rule and Pointed Graph

Some grammar languages provide some starting point for validating documents or generating forms. In Shape Expressions, the starting point is specified by the start keyword.

start = <IssueShape>

This directive says to start with the <IssueShape>, which is really <http://base.example/#IssueShape> because of the BASE directive above.

It is not necessary to identify a particular node in the graph for validation operations. Nor is it necessary to provide a starting point for all opperations. For instance, generating a sequence of forms obviously needs to start somewhere, but some documents can be validated by optimistially testing each shape expression against each node in the graph. This exhaustive search is more expensive and raises the possibility that a document validates in a way that the author of the document did not intend. This document treats the more constrained scenario with a starting point in both the graph and the schema.

Labeled Shape Expression

A shape expression is a labeled pattern for a set of RDF Triples with a common subject. Syntactically, it's a pairing of a label, which is an IRI or a blank node, and a rule inside a pair of "{" "}". Typically, this rule is a conjunction of constraints separated by ',':

<IssueShape> {
    ex:state (ex:unassigned ex:assigned),
    ex:reportedBy @<UserShape>,
    ex:reportedOn xsd:dateTime,
    ( ex:reproducedBy @<EmployeeShape>,
      ex:reproducedOn xsd:dateTime      )?,
    ex:related @<IssueShape>*
}

(Links in the above example detail the interpretation of the above constraints.)

The following UserShape is referenced by the IssueShape above. Where the above labeled shape expression describes the data related to issues in some issue tracker, this captures the information about the users of that system.

<UserShape> {
    (foaf:name xsd:string
     | foaf:givenName xsd:string+,
       foaf:familyName xsd:string),
    foaf:mbox IRI
}

The EmployeeShape below is referenced by the IssueShape above and included here for completeness. It does not introduce any new features of the language.

<EmployeeShape> {
    foaf:givenName xsd:string+,
    foaf:familyName xsd:string,
    foaf:phone IRI+,
    foaf:mbox IRI
}

Cardinality

The <IssueShape> example also includes a GroupRule with a cardinality of 0 or 1:

    ( ex:reproducedBy @<EmployeeShape>,
      ex:reproducedOn xsd:dateTime      )?

As explained above, this requires that the data have either neither or both of those triples.

Cardinality constraints may appear on ArcRules and GroupRules. They may be expressed as one of (?, +, *) or as one or two integers in {}s. If there is only one number in {}s, the minimum cardinality is that number and the maximum is unconstrained. An employee record which permitted from one to three given names would look like

    foaf:givenName xsd:string{1,3}

Inheritance and Inclusion

It is frequently useful to reuse or extend a shape. For instance, if both the <UserShape> and <EmployeeShape> permitted the same alternatives for specifying a name and email address, these could be factored into a separate shape called a <PersonShape>:

<PersonShape> {
    ( foaf:name xsd:string
      | foaf:givenName xsd:string+,
        foaf:familyName xsd:string
    ),
    foaf:mbox IRI
}

<UserShape> {
    & <PersonShape>
}

<EmployeeShape> {
    & <PersonShape>,
    foaf:phone IRI+
}

In this example, the <UserShape> provides no additional constraints beyond those of the included <PersonShape>.

We may have several derivatives of <PersonShape>, any of which could provide an <IssueShape>'s ex:reportedBy value. We can signify this by changing <IssueShape> to have ex:reportedBy @<PersonShape> and define sub-shapes of <PersonShape>. This is done with an inclusion directive before the shape definition. We may not want the base <PersonShape> to satisfy any ValueReferences directly, instead requiring only derivates of <PersonShape>. This is accomplished by labeling <PersonShape> VIRTUAL per the hierarchy example:

<IssueShape> {
    ex:state (ex:unassigned ex:assigned),
    ex:reportedBy @<PersonShape>
    # ...
}

VIRTUAL <PersonShape> {
    ( foaf:name xsd:string
      | foaf:givenName xsd:string+,
        foaf:familyName xsd:string
    ),
    foaf:mbox IRI
}

<UserShape> & <PersonShape> {
    # additional User properties
}

<EmployeeShape> & <PersonShape> {
    foaf:phone IRI+
    # additional Employee properties
}

Should there be a "greedy" directive to accept only the variant which touches the most triples?

Semantic Actions

The <IssueShape> example above includes both ex:reportedOn and ex:reproducedOn dateTimes. It would be reasonable in the interest of data quality to ensure that the ex:reproducedOn dateTime, if present, were temporally after the ex:reportedOn dateTime. While ShEx itself has no built-in functionality for comparing dateTimes, specific extensions may offer that functionality. The example below (failed semantic action validation) includes semantic actions to test date order in either Javascript or SPARQL:

    ex:reportedOn xsd:dateTime
        %js{ report = _.o; return true; %},
    (ex:reproducedBy @<EmployeeShape>,
     ex:reproducedOn xsd:dateTime
        %js{ return _.o.lex > report.lex; %}
        %sparql{ ?s ex:reportedOn ?rpt . FILTER (?o > ?rpt) %}
    )

Semantic actions may also be used to generate schema-specific parsers or tools. Below is an excerpt of a tool which uses a DOM tree to translate the the Issue example into an XML document:

<IssueShape> {
    ex:state (ex:unassigned ex:assigned)
        %js{ doc = _.createDocument('http://ex.example/xml', 'Issue', undefined);
             issue = doc.documentElement
             issue.setAttribute('id', _.s.lex.substr(17));
             state = doc.createElementNS('http://ex.example/xml', 'state');
             state.textContent = _.o.lex.substr(17);
             issue.appendChild(state); %},
    ex:reportedBy @<UserShape>,
    …
} %js{ console.log(new XMLSerializer().serializeToString(doc)); %}

This example relies on a particular invocation for semantic actions, but illustrates the power in the extensibility mechanism.

Language Summary

The IssueShape tutorial above is oriented towards a particular use case where the schema will use a very explicit set of predicates and accept no others. Shape Expressions is also useful for controlling describing interfaces or graph patterns where any predicates are allowed except those in controlled namespaces. For example, some systems like Annotea reserved the assertion of dc:creator arcs for the system to maintain provenance information. The language summary below includes language features to describe such an interface.