# Differentiating between undefined and null

**URL:** <https://softwaremill.community/t/differentiating-between-undefined-and-null/154>\
**Category:** tapir\
**Created:** [March 21, 2023, 6:26pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154 "2023-03-21T18:26:49Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jason\_Xiong](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/jason_xiong/32/67_2.png) [@Jason\_Xiong](https://softwaremill.community/u/Jason_Xiong)\
**Post date:** [March 21, 2023, 6:26pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/1 "2023-03-21T18:26:50Z")

</div>

Wondering if it’s possible to differentiate between undefined and nulls in Tapir schema. I’m attempting to implement something based on [Json Merge Patch](https://www.rfc-editor.org/rfc/rfc7386), which has different behavior depending on if the incoming field is undefined vs null. I have a custom `circe` type defined roughly as:

```auto
sealed trait JsonMaybe[+A]
case class JsonSome[+A](value: A) extends JsonMaybe[A]
case object JsonNull extends JsonMaybe[Nothing]
case object JsonUndefined extends JsonMaybe[Nothing]

```

and now I need to create a custom Tapir schema that supports this. It doesn’t work by default since tapir thinks any field defined as `JsonMaybe` instead of `Option` is required, so I’m trying to figure out a generic way to make `JsonMaybe` also be `optional` in the eyes of Tapir. Does this require implementing my own `SchemaType`, like `SchemaType.SOption`?

Edit:  
Adding a bit more context:  
The confusing thing here is that the circe decoder works differently when by itself versus when integrated with tapir. For example

```auto
case class Foo(maybe: JsonMaybe[Unit])

decode[Foo]("{}") // correctly decodes to JsonUndefined

@endpointInput("foo")
case class FooInput(
  @jsonbody
  foo: Foo
)

// Sending a request with body "{}" will result in "Missing required field at 'maybe'"

```

---

<div class="post-metadata">

**Author:** ![adamw](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/adamw/32/28_2.png) [@adamw](https://softwaremill.community/u/adamw)\
**Post date:** [March 22, 2023, 9:42am UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/2 "2023-03-22T09:42:33Z")

</div>

As for the confusing part, what matters is the place from which you invoke `Schema.derived` (or run auto-derivation). Is your customised circe codec visible there?

And as for the original question - how would that be represented in the OpenAPI Schema? A field which can be `null`/`undefined` is optional, but how else would it influence the documentation?

---

<div class="post-metadata">

**Author:** ![JonasB](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/jonasb/32/128_2.png) [@JonasB](https://softwaremill.community/u/JonasB)\
**Post date:** [March 31, 2023, 1:30pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/3 "2023-03-31T13:30:54Z")

</div>

I’m trying to achieve the same thing.

On the circe side, I’ve tried the solution by Travis Brown on [Stack Overflow](https://stackoverflow.com/a/60005696). It seems to work fine, but I can’t get the OpenAPI specification right.

In OpenAPI 3.0.3 there’s `"nullable": true` as well. I guess the field should be both nullable and optional. Is that possible with tapir?

---

<div class="post-metadata">

**Author:** ![adamw](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/adamw/32/28_2.png) [@adamw](https://softwaremill.community/u/adamw)\
**Post date:** [April 22, 2023, 8:08am UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/4 "2023-04-22T08:08:04Z")

</div>

You can configure it globally only, to mark all optional fields as nullable, by customising the [OpenAPI options](https://tapir.softwaremill.com/en/latest/docs/openapi.html#options), see: [tapir/OpenAPIDocsOptions.scala at dc8dd857a1f4b2d5b0498d787857f36619b0cac5 · softwaremill/tapir · GitHub](https://github.com/softwaremill/tapir/blob/dc8dd857a1f4b2d5b0498d787857f36619b0cac5/docs/openapi-docs/src/main/scala/sttp/tapir/docs/openapi/OpenAPIDocsOptions.scala#L9)

Would that work for you?

---

<div class="post-metadata">

**Author:** ![JonasB](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/jonasb/32/128_2.png) [@JonasB](https://softwaremill.community/u/JonasB)\
**Post date:** [May 30, 2023, 2:53pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/5 "2023-05-30T14:53:17Z")

</div>

Sorry for the late reply.

Unfortunately that doesn’t work for the use case I have.

There are lots of other types that I don’t want to change, and then one for “FooPatchDto” where I’d like all of the fields be optional and nullable. It would need to be something that’s configurable per field or on the type.

---

<div class="post-metadata">

**Author:** ![adamw](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/adamw/32/28_2.png) [@adamw](https://softwaremill.community/u/adamw)\
**Post date:** [May 30, 2023, 3:34pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/6 "2023-05-30T15:34:04Z")

</div>

So I think the option you would have right now is to customise the schema using one the methods [described here](https://tapir.softwaremill.com/en/latest/endpoint/schemas.html#customising-derived-schemas).

In a nutshell, you can add the `@customise(Schema => Schema)` annotation to a class, and in the function recursively check if the schema is optional, and if so, set the `nullable` property to `true` as well. Or you can customise the value obtained using `Schema.derived` and assign it to an implicit value.

---

<div class="post-metadata">

**Author:** ![JonasB](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/jonasb/32/128_2.png) [@JonasB](https://softwaremill.community/u/JonasB)\
**Post date:** [June 1, 2023, 8:09am UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/7 "2023-06-01T08:09:09Z")

</div>

I can’t find a property called “nullable” on any type. Is there one, or a more generic data structure where I can put in any key-value pair and have it be rendered as part of the property in the generated Open API yaml?

I tried using `@customise` but couldn’t understand how to use it in a good way.  
I added this code (`UpdateOrRemoveOrIgnore` is my equivalent of Jason’s `JsonMaybe`):

```auto
object UpdateOrRemoveOrIgnore {
  implicit def tapirSchema[T: Schema]: Schema[UpdateOrRemoveOrIgnore[T]] =
    implicitly[Schema[T]].asOption
      .as[UpdateOrRemoveOrIgnore[T]]
      .description(
        "Update the field if it's sent in with a value, remove it if sent in with 'null', ignore it otherwise"
      )
  // circe code here
}

```

And with that change, the documentation rendered in ReDoc looks a lot nicer.

For this case class

```auto
case class FooPatchDto(someOptionalParameter: UpdateOrRemoveOrIgnore[Long])

```

the generated OpenAPI yaml looks like this:

```auto
    FooPatchDto:
      type: object
      properties:
        someOptionalParameter:
          type: integer
          format: int64

```

If I can add `nullable: true` to it, so it would look like:

```auto
    FooPatchDto:
      type: object
      properties:
        someOptionalParameter:
          type: integer
          format: int64
          nullable: true

```

it would be perfect.

---

<div class="post-metadata">

**Author:** ![adamw](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/adamw/32/28_2.png) [@adamw](https://softwaremill.community/u/adamw)\
**Post date:** [June 7, 2023, 8:40am UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/8 "2023-06-07T08:40:19Z")

</div>

Indeed, you are right. The `nullable` property is only available on `sttp.apispec.Schema` type - which represents the JSON Schema data structure. This is generated from `sttp.tapir.Schema[T]`, which describes a type. The tapir schema doesn’t have the nullable property, and it can currently only be set globally.

As a work-around, you can modify the `OpenAPI` data structure that the interpreter returns. It’s not the prettiest, but does the job. For example:

```plaintext
  case class Test(s: Option[String])

  test("x") {
    import com.softwaremill.quicklens._
    val e = endpoint.in(jsonBody[Test])
    val openapi = OpenAPIDocsInterpreter().toOpenAPI(List(e), Info("Fruits", "1.0"))
    val openapi2 = openapi.modify(
      _.components.each.schemas.at("Test").eachRight.when[sttp.apispec.Schema
       .properties.at("s").eachRight.when[sttp.apispec.Schema]
       .nullable
    ).setTo(Some(true))
    val actualYaml = openapi2.toYaml
    println(actualYaml)
  }

```

As for a more user-friendly fix, I think we should have an option to customise the final, generated JSON Schema (which is part of the `OpenAPI` object). We could set an attribute on the tapir-schema, with a function which would be called by the OpenAPI interpreter.

Can you create an issue to implement this?

---

<div class="post-metadata">

**Author:** ![JonasB](https://dub1.discourse-cdn.com/flex005/user_avatar/softwaremill.community/jonasb/32/128_2.png) [@JonasB](https://softwaremill.community/u/JonasB)\
**Post date:** [June 9, 2023, 12:59pm UTC](https://softwaremill.community/t/differentiating-between-undefined-and-null/154/9 "2023-06-09T12:59:25Z")

</div>

Thanks for the help, Adam!

I got it working with this change:

```auto
object UpdateOrRemoveOrIgnore {
  // The description is also used as a marker to find the fields that we want to set as "nullable: true" in the
  // generated Open API yaml
  private val UpdateOrRemoveOrIgnoreMarker = "Update the field if it's sent in with a value, remove it if " +
    "sent in with 'null', ignore it otherwise"

  def modifyToMakePatchesWork(originalOpenApi: OpenAPI): OpenAPI = {
    originalOpenApi
      .modify(
        _.components.each.schemas.each.eachRight
          .when[sttp.apispec.Schema]
          .properties
          .each
          .eachRight
          .when[sttp.apispec.Schema]
      )
      .using((s: sttp.apispec.Schema) => {
        if (s.description.contains(UpdateOrRemoveOrIgnoreMarker)) {
          s.copy(nullable = Some(true))
        } else s
      })
  }

  implicit def tapirSchema[T: Schema]: Schema[UpdateOrRemoveOrIgnore[T]] =
    implicitly[Schema[T]].asOption
      .as[UpdateOrRemoveOrIgnore[T]]
      .description(
        UpdateOrRemoveOrIgnoreMarker
      )

  // circe code here
}

```

and passing `customiseDocsModel = UpdateOrRemoveOrIgnore.modifyToMakePatchesWork` to `SwaggerInterpreter` and `RedocInterpreter`.

Issue created! [[FEATURE] Support customizing the generated OpenAPI JSON schema · Issue #2953 · softwaremill/tapir · GitHub](https://github.com/softwaremill/tapir/issues/2953)
