List with Filters, Sorting and Paging

The shop’s back office lists orders: only the pending ones, the largest first, twenty at a time. A client says so in the query string, and the method’s query declaration decides how much of that the client controls: a route can leave the selection open, bound it, or fix it.

§The parameters a list takes

A method that maps an operation with a query accepts the standard parameters. GET /store/orders/ is one: Expose It over HTTP maps it to the inherited enumerate of store.orders, the component Create a Component writes, beside POST: create and anonymous: true, which the fragments here leave out.

# components/store.orders/manifest.toa.yaml
exposition:
  /:
    # …
    io:output: [id, customer, status, total]
    GET: enumerate

GET /store/orders/?criteria=total>1000&sort=total:desc&omit=20&limit=10 HTTP/1.1

omit and limit apply to a method whose operation works on a list — an observation, a transition or an effect of scope entries, such as enumerate. A method that works on one entity refuses them with 400.

As a scenario, written with the steps of Write a Test and run with npm run features while sys is running (npm run sys):

# features/orders.feature
  Scenario: List a customer's pending orders, largest first
    When the request is sent:
      """
      POST /store/orders/ HTTP/1.1
      content-type: application/yaml

      customer: b70a7dbca6b14a2eaac8a9eb4b2ff4db
      items: []
      total: 4200
      """
    Then the response is received:
      """
      201 Created
      """
    When the request is sent:
      """
      GET /store/orders/?criteria=customer==b70a7dbca6b14a2eaac8a9eb4b2ff4db;status==pending&sort=total:desc&limit=1 HTTP/1.1
      accept: application/yaml
      """
    Then the response is received:
      """
      200 OK

      - customer: b70a7dbca6b14a2eaac8a9eb4b2ff4db
        status: pending
      """
    When the request is sent:
      """
      GET /store/orders/?limit=1000 HTTP/1.1
      accept: application/yaml
      """
    Then the response is received:
      """
      400 Bad Request
      """

An operation that declares query: false takes no query string at all:

HTTP/1.1 400 Bad Request

"Query parameter 'limit' is not allowed"

§Criteria

criteria is RSQL. Conditions are joined with ; for and and , for or:

GET /store/orders/?criteria=status==pending;total>=1000 HTTP/1.1
GET /store/orders/?criteria=status==pending,status==approved HTTP/1.1

A value is read as what the property holds. A value that cannot be read so is refused:

GET /store/orders/?criteria=total>abc HTTP/1.1

HTTP/1.1 400 Bad Request

"Criteria selector 'total' takes a number, and 'abc' is not one"

A string property takes whatever is written.

An unquoted null or undefined means no value: approved==null selects the orders that hold no approved — never set, or set to null — and approved!=null those that hold one. It compares with ==, !=, =in= and =out=; any other operator with it is 400. Quoted, it is text: note=="null".

§Properties inside an object

A condition names a property inside an object by the path to it. Say an order holds where it is delivered:

# components/store.orders/manifest.toa.yaml
entity:
  properties:
    # …
    delivery:
      type: object
      properties:
        city: { type: string }
        floor: { type: integer }

GET /store/orders/?criteria=delivery.city==Porto;delivery.floor>3 HTTP/1.1

The value is read as what the property at the end of the path holds, here a string and an integer. A path that names nothing the entity declares is refused:

GET /store/orders/?criteria=delivery.street==Flores HTTP/1.1

HTTP/1.1 400 Bad Request

"Criteria selector 'delivery.street' is not defined"

sort and projection take a path the same way, sort=delivery.city, and an index is declared on one as on a property.

A path through an array of objects, such as items.quantity>=10, selects the orders where some item meets the condition. Each condition is met on its own, by any item.

§Fixing the criteria

A route that declares criteria selects with it, and the client adds nothing:

exposition:
  /pending:
    io:output: [id, customer, total]
    GET:
      endpoint: enumerate
      query:
        criteria: status==pending

GET /store/orders/pending/?criteria=total>1000 HTTP/1.1

HTTP/1.1 400 Bad Request

"Query criteria is closed"

§Opening the criteria

Criteria that end with ; are open: the client’s criteria are added to the route’s, joined with and.

      query:
        criteria: status==pending;    # open

GET /store/orders/pending/?criteria=total>1000 now selects (status==pending);(total>1000). The client can narrow the route’s selection and never widen it.

§Criteria and route variables

A route variable other than id is a condition of its own, put in front of the route’s criteria and joined with and:

exposition:
  /of/:customer/pending:
    io:output: [id, total]
    GET:
      endpoint: enumerate
      query:
        criteria: status==pending

GET /store/orders/of/b70a…f4db/pending/ selects (customer==b70a…f4db);(status==pending).

To join the variable with or instead, start the criteria with a comma:

      query:
        criteria: ,status==pending

which selects (customer==b70a…f4db),(status==pending). A leading ; states the default explicitly. POST is the exception: its route variables go to the input, see Map Methods to Operations.

§Sort

GET /store/orders/?sort=total:desc;CREATED:asc HTTP/1.1

Ascending is the default: sort=total is sort=total:asc. CREATED is one of the system properties every stored object carries.

A route that declares sort orders by it. Ending the value with ; opens it, and the client’s sort is appended after the route’s:

      query:
        sort: status:asc;    # then by whatever the client asks

GET …?sort=total:desc then orders by status:asc;total:desc.

Declare a default and the bounds a client may move within:

exposition:
  /:
    io:output: [id, status, total]
    GET:
      endpoint: enumerate
      query:
        sort: CREATED:desc
        limit:
          value: 20
          range: [1, 50]
        omit:
          range: [0, 500]

value is what applies when the request says nothing; without it the lower bound of the range is used. A request outside the range is refused:

HTTP/1.1 400 Bad Request

"Query limit must be between 1 and 50 inclusive"

A number alone is a constant the client cannot change:

      query:
        limit: 5

Where nothing is declared the defaults are:

omit:
  value: 0
  range: [0, 1000]
limit:
  value: 10
  range: [1, 100]

They hold for a route to a transition or an effect over entries as well: a PUT that carries no limit changes ten entities, never the whole collection.

omit needs an order. A request with omit to a route that neither declares nor receives a sort is answered 400. Declare sort on every route a client pages through, as above.

§A fixed query

Putting these together, a route can be a complete, closed question:

exposition:
  /top:
    io:output: [id, customer, total]
    GET:
      endpoint: enumerate
      query:
        criteria: status==approved
        sort: total:desc
        limit: 10
  /latest:
    io:output: true
    GET:
      endpoint: observe
      query:
        sort: CREATED:desc

GET /store/orders/top/ returns the ten largest approved orders, and GET /store/orders/latest/ the newest order.

§Identifier

One entity is usually selected by a route variable named id. A client may also send it as a parameter, where the route has no such variable:

exposition:
  /order:
    io:output: true
    GET: observe

GET /store/orders/order/?id=4c4759e6f9c74da989d64511df42d6f4 HTTP/1.1

§Text search

For an entity with a text index (declared in a migration), a method can accept a search parameter. It is off unless declared. The catalogue is a component of its own, store.catalog, whose products have a title and a price:

# components/store.catalog/manifest.toa.yaml
exposition:
  /:
    io:output: [id, title, price]
    GET:
      endpoint: enumerate
      query:
        search: true

GET /store/catalog/?search=copper+kettle HTTP/1.1

A search sent to a method that does not declare it is 400.

§Projection

projection lists the entity properties an observation reads from the storage. The operation receives those, the id and the system properties, and nothing else:

exposition:
  /:
    io:output: [id, status, total]
    GET:
      endpoint: enumerate
      query:
        projection: [status, total]

Use it for wide entities of which a route needs little: the properties that are not listed are never loaded. A method without a projection reads the whole entity.

• id is always read; naming it in the projection is refused.
• Only a method mapped to an observation may declare one. Declared for any other type, the component does not start.
• A projection decides what is read. What the client receives is still decided by io:output.

§Parameters of the operation

Query-string parameters other than the standard ones are refused, unless the method declares them as parameters. Declared parameters are not part of the query: they are put into the operation’s input, which must be an object.

# components/store.catalog/manifest.toa.yaml
exposition:
  /quote:
    io:output: true
    GET:
      endpoint: quote          # a computation taking { product, quantity }
      query:
        parameters: [product, quantity]

GET /store/catalog/quote/?product=kettle&quantity=3 HTTP/1.1

A parameter that is not declared is refused:

HTTP/1.1 400 Bad Request

"Query parameter 'foo' is not allowed"

§Streams of a collection

A method mapped to an operation of scope stream answers the selected entities and then a token to continue from (Read a Large Collection); every component with an entity inherits one, named stream. Its query follows different rules:

• limit is the size of a window. A stream route that declares no limit answers the whole collection in one reply and takes no limit from the client.
• omit is refused. The next window is read with the token.
• token continues from where a previous reply ended.
• stop ends the read after the first window; the token then continues with changes only.
• sort, beside limit or token, takes id and CREATED only.

# components/store.orders/manifest.toa.yaml
exposition:
  /feed:
    GET:
      endpoint: stream
      io:output: [id, status, total, VERSION]
      query:
        criteria: status==pending
        limit: { value: 100, range: [1, 500] }

GET /store/orders/feed/ HTTP/1.1
GET /store/orders/feed/?token=eyJ2IjoxLCJwIjoi... HTTP/1.1
GET /store/orders/feed/?sort=CREATED:desc&limit=20&stop HTTP/1.1

token and stop sent to a method that is not a stream are 400. A token under criteria or an order other than its own is 400; a token the storage can no longer continue from is 410 Gone, and the client reads the collection again without one. The shape of the reply and how a client follows it is in Stream Through the API.

§Reference

Keys of a method’s query:

Parameters of a request:

The version a write is conditional on travels in the if-match header, not in the query string; see Prevent lost updates.