dotnet core api behind proxy

I manage the central API ingress for company applications, which means I am interfacing with many different teams, codebases, and languages to enable a single api.company.com entrypoint which dynamically routes to multiple different backing microservices.

For nearly every team and application I work with, this is as easy as implementing a new location block on the environment-specific server context, maybe with some extra headers / configs to suit their specific needs before proxying the request on to their microservice.

However recently I was tasked with connecting directly with a DotNet Core API running in ECS Fargate which exhibited very interesting issues when serving behind a path-rewriting proxy server.

The team in question previously had been using API Gateway as the HTTP wrapper for their microservice logic, but since our company handles large media files, the hard 15 minute timeout on Lambdas and 30s timeout on API Gateway made their traditional workflow impossible to follow.

By having the team write an actual HTTP API server (in their language of choice - DotNet) and then serving this from within a Docker container hosted in ECS Fargate, I was able to mitigate the timeout issues seen in Lambda and API Gateway. However now I had to work directly with a DotNet API, which I quickly learned was opening a can of worms of its own.

There is an active bug in AspNetCore which causes a litany of issues when DotNet APIs are hosted behind a path-rewriting proxy server. This is the case for our architecture - api.company.com/teamA/upload will point to the microservice for teamA's upload service, but api.company.com/teamB/download will point to the microservice for teamB's download service.

It is through the use of the path appended to the central API system that the respective microservice is called, and DotNet Core does not play well with such systems. DotNet Core Hosting makes the faulty assumtion that it is always served at the root of whatever server is communicating with the backing DotNet application.

While [nearly] every other HTTP server tool and framework can handle such proxied requests and responses, DotNet will not be able to respond / handle requests unless the server path that is making the request exactly matches the path(s) defined in the Controller Router definitions.

While a somewhat-of-a-hack app.PathBase("/path-base") middleware does exist, this seemingly only affects the root / static routes, and does not affect subsequent / called Controller routes (the root cause of the open issue / bug mentioned above).

After days of debugging and hours of "it's your DotNet code" and "no it's your server code" back and forth with the team in question, I came to a working - but hacky - solution to this issue, until the aforementioned DotNet bug is fixed, at least.

While DotNet is not able to handle dynamic paths appended in a proxy server, if a proxy route is defined for every unique controller route, DotNet is happy.

A conventional proxy server will define a root path for a request and proxy all paths appended to this request on to the backing service, however this throws 404 errors in DotNet since it is assuming all Routes are served at the root (/) of the server.

But by explicitly defining a hard request path mapping to the same DotNet Route path, DotNet responds as expected. This, of course, requires a unique location block for every single Route path defined in the DotNet API code.

To mitigate this, I created a script which, on each build and deploy of their DotNet API application, finds all Route definitions in the code, and then creates a unique location block mapping the path-rewriting proxy path to that specific route on the ECS NLB, using the root proxy location block as a template.

Is it a hack? Definitely. Does it cause a fair bit of [dynamically generated] code duplication? For sure. But does it ensure the development team can roll out iterative updates to their microservices without me having to manually modify ops-code? You bet.

And that's a win in my book.

last updated 2024-10-03