How to get access to the request payload in Tapir/ZIOHttp DefaultServerLog?

tapir
1

We build a REST microservice with Scala 3, ZIO 2, ZIO logging and Tapir.

For context specific logging we want to use the MDC and set an attribute there which is taken from the request payload.

Is it possible to get access to the request payload in DefaultServerLog to extract the MDC attribute and then use it for ZIO logging feature MDC logging, i.e. create a LogAnnotation from the extracted attribute, so it will also be logged by all DefaultServerLog methods (doLogWhenHandled etc.)
Currently it works for our own log statements, but not for those of Tapir/ZIO-HTTP.

2

This is usually problematic as the body of the request is a stream of bytes, which is read from the socket as it arrives. That is, the request isn’t loaded into memory by default.

You can work-around this by reading the whole request into memory using serverRequest.underlying.asInstanceOf[zio.http.Request].body.asArray, extracting the required info and enriching the fiber-locals appropriately. You might also need to substitute the Request with a copy, which has the body provided as a byte array (in a “strict” form), so that the “proper” body parser doesn’t try to re-read from the network (where nothing will be available).

However, this has some downsides: the body will be parsed twice (once by your interceptor, once by the parsing that’s defined later); and it will be read into memory (which might be problematic if you don’t have a limit on the size of the body).

1 Like
3

Thank you very much for this excellent response,Adam!

4

@adamw If we would give the attribute as an HTTP header, would this simplify the situation (as we don’t need to parse the request body).

How would we get access to the HTTP header from DefaultServerLog, so we can provide it as a MDC attribute?

5

Is the correlation id set into a fiber-local? If so, I’d prepend a RequestInterceptor to CustomizeInterceptors which extracts its value and sets it. Sth like:

new RequestInterceptor[F] {
  override def apply[R, B](
      responder: Responder[F, B],
      requestHandler: EndpointInterceptor[F] => RequestHandler[F, R, B]
  ): RequestHandler[F, R, B] =
    new RequestHandler[F, R, B] {
      override def apply(request: ServerRequest, endpoints: List[ServerEndpoint[R, F]])(implicit
          monad: MonadError[F]
      ): F[RequestResult[B]] = {
        val cid = request.header("Correlation-Id").getOrElse(newCorrelationId())
        correlationIdLocal.set(cid) *> requestHandler(EndpointInterceptor.noop)(request, endpoints)
      }
    }
}
1 Like
6

I’ve created an example of how this might work: Zio netty interpreter improvements, logging example by adamw · Pull Request #2677 · softwaremill/tapir · GitHub

Note that it uses a (simple) helper function in RequestInterceptor, which isn’t yet released.

1 Like
7

Thank you so much Adam. As I am still newish to ZIO, I tried yesterday to get my head around what you wrote (FIberRef etc). So your example is highly appreciated, I will look into it!

1 Like
8

Works like a charm, Adam. Thank you so much. Still a bit confused how the basic ZIO.logAnnotate interacts with ZIO logging, but that is not Tapirs problem.

9

You have to read the code from the point of view of an interpreter. The interpreter gets a ZIO value and peels of subsequent layers. So ZIO.logAnnotate(annotation)(anotherZIO) will create a description of a computation, which is +/- a LogAnnotate case class, containing the nested computation to execute.

So when the interpreter sees a LogAnnotate description, it will modify its current “logging state” to include that annotation in any logs, that are created as part of the nested computation. It then proceeds to evaluating the nested computation.

That’s of course more of a high-level view, but at some level matches the reality :wink:

1 Like
10

Thank you very much for your explanation. That helps a lot!