Reg. validation of endpoints against provided schema

tapir
1

Regarding this issue:

I could really use such a “static validator”.
Thing is, that I don’t think it’s feasible to fully validate “statically”.

By “static” I mean given 2 definitions, whether they’re in openAPI format, scala tAPIr code, or anything else, I don’t think we can always prove they’re “isomorphic”.

Same API can be described in multiple ways.
For instance, an API that responds with a body that references a schema like:

components:
  schemas:
    User:
      type: object
      properties:
        email:
          type: string
        name:
          type: string
      required:
        - email
        - name

can also be documented in a more relaxed way, e.g:

components:
  schemas:
    User:
      type: object
      additionalProperties:
        type: string

The definition still holds, and correct for all possible outputs.
In Scala, it would be equivalent to use a case class of String members: case class User(email: String, name: String) and replace it with a more permissive type Map[String, String] which would still be correct.

When you think about it, you could document every API with Any as return type, and it will be correct.

Other corner cases are different uses of oneOf, anyOf, allOf, not…
is

oneOf:
  - string
  - boolean

different than:

not:
  oneOf:
    - integer
    - number
    - array
    - object

?

Point is, validating 2 schemas equivalency is hard.
It’s easier to validate that a specific request/response does not violate a given contract.

When I researched this, I came across this:
https://bitbucket.org/atlassian/swagger-request-validator
(and also: Bitbucket )

My goal was to use openapi contract generated from a non-scala service (python) to generate a scala client (zio).
I ended up writing the contract manually, for exactly the same reasons mentioned above.
The openapi contract generated from python was very general, used allOf to denote inheritance with fields, while I needed to be specific, and generate sealed traits with oneOf instead.

So now, the problem was: how do I validate that the manually written contract stays correct?

You can see why I take interest in this issue.
Any ideas on how to make it usable when the schemas are actually different, but still describe the same requests/responses in a different way?

2

You’re certainly rising valid concerns, however I think the scope of the feature would be narrower - and we’d have to properly document it. The use case that I’d like to cover is that a schema is generated for a server, published, and then we want to verify that we don’t break the existing contract. So we never had the ambition of providing a general-purpose OpenAPI equivalence verifier, which would be hard for the reasons provided (“subtyping”, coproducts etc.).

But given the narrower scope as described above (and in the issue), it’s enough to verify that the schemas are identical (not equivalent) in the “parts that matter” (e.g., descriptions do not matter).

I think your use-case is different, so probably out of the scope of this feature in particular.

3

makes sense. I hoped maybe I’d be able to use it, but perhaps not.
Do you think that an equivalency verifier for tAPIr endpoints is something that might be provided in the future?

Perhaps it would not be a “proof”, but rather a “check”, kind of like property checks tests…
Speaking of property checks, it gives the idea of some sort of Gen that takes an Endpoint, and generate valid requests & responses, which then can be validated against a given OpenAPI spec.

I could really use such a tool to write verification tests for a client to any service exposing an OpenAPI spec.

WDYT?

4

I think that we might start with an “identity” check, and then it could be expanded to be more “intelligent”, allowing for greater flexibility in verifying subtyping & coproduct rules. It’s a complex problem, as you wrote, so probably needs gradual work.

I guess having an Endpoint instance and generating requests & responses would be doable, but that’s another feature (though very interesting, also from a security standpoint! Maybe create an issue for it so it doesn’t get lost?). But I’d attempt at implementing it without going through the (lossy) OpenAPI format.

1 Like
5

done!

I’m not sure how hard it would be to implement this, but I would definitely be happy to use such a tool!
Thanks :heart: