URI decoding when making sttp request

sttp client
1

Hey all!
I ran into an issue where requests to a certain website were failing because the uri decoding that sttp does causes the verification to fail.

eg.

scala> import sttp.client4._
                                                                                                                                                 
scala> uri"http://example.com/query=query1%2Cquery2"
val res0: sttp.model.Uri = http://example.com/query=query1,query2

Looking at the documentation, it seems this was intentional. I was just wondering if anyone knows the rationale behind why sttp does this? It seems if we want to just pass the URI to the request we would need to change the encoding rules for the segments? Ideally, I would like to be able to pass the URI to the request without having to change them.

I see in the documentation it mentions this might change in a future release, any plans on this?

2

The reason is that the Uri class is designed to store the parsed (decoded) URI. So there’s no way in sttp to represent an uri as a “raw string”.

You can, however, provide custom encoding functions for all parts of the uri (path, query, fragment, etc.), as described in the documentation.

Even though sttp4 is a major release, which can break binary compatibility, it shares the same sttp-model version, so no changes in the Uri class will happen.

1 Like
3

Thanks Adam!

So we would need to do something like

uri"$url".pathSegmentsEncoding(encode(someCharSet)).querySegmentsEncoding(Uri.QuerySegmentEncoding.All)

Do you happen to know if there’s an example somewhere for the encoding rules we would need to apply for it to just reflect the raw string passed to the uri? Something that might mimic the behaviour expected from the Java URI class. I’m a bit worried that I might make a mistake when adding custom encoding.

4

Well it’s hard to say without knowing how the original was encoded.

Generally, the defaults in sttp’s Uri class follow the RFCs. I think Java encodes “too much”, that is some characters which don’t need to be encoded according to the specification are encoded anyway. This corresponds to QuerySegmentEncoding.All.

1 Like