This is the story of an organization that seems to have forgotten basic software engineering concepts. This organization does exists and has these exact problems, but it is also a generic story about similar organizations that abandon engineering excellence for their own peril.
We have forgotten how to run distributed applications
Long ago, the organization ran mostly on Weblogic. As a rule, there were always two Weblogic servers that hosted any given application. What we have forgotten is that the Weblogic servers were configured to enable distributed transactions, so even when two instances of the same application accessed the same data, there was a high degree of assurance that there will be no data corruption nor race conditions.
Fast forward to now, the organization migrated to Kubernetes, kept the old rule of having two instances of every application, but forgot why it was possible previously.
The organization being highly compartmentalized, developers did their their own migration from EJB to Spring and then Spring Boot where the likes of JPA and ehcache created lots of state within the application instances… which, given the absence of an application server that synchronized state among instances, resulted in spurious race conditions that are often resolved via manual reboots because… we have forgotten how to make distributed apps.
We have forgotten session cookies
I do not know what was the reason to get rid of session cookies. Maybe the fear of GDPR violations or because the organization invested in an API Gateway designed around JWT tokens.
Still, the organization forgot about session cookies and bet on an application design where
- frontend applications in Angular are publicly accessible on their own domains, like gui.serviceprovider.com.
- Java backend is accessed via the API Gateway on separate domains like backend.serviceprovider.com that are protected by the
Authorization:header that should contain a signed JWT.
This immediately created subtle CORS problems that were dealt with valiantly but successfully. They still make life harder in edge cases, e.g. when interfacing with mobile apps. Less obvious problems appeared later. For instance, many applications exposed files via the backend.
Now, these files could not be downloaded via the browser, by lack of session cookies. The JavaScript frontend had to authenticate via OAuth 2.0, obtain a properly signed JWT token, download a file, then forward it to the user. Or worse, when the file had to be displayed inline, the frontend JavaScript code created a Blob object, then asked the browser to render it.
Downloading a file from the backend became a challenge in its own right. The API Gateway imposed JSON APIs for all calls, so what could be a simple GET request to an octet/stream became a multipart/form-data response, with the binary accompanied by a JSON that contained the file name and the file description, because… we forgot about Content-Disposition headers either.
Users are now required to authenticate every time they switch from one single-page application to another, because JWT tokens live only in the context of the JavaScript execution, thus rendering the use of Single Sign-On OAuth 2.0 workflow totally moot.
Screen readers and many kinds of web accessibility are barely useable, all that… because we have forgotten about session cookies.
How to unforget?
There is clearly a failure of engineering leadership. I am still puzzled why. Because of lack of competition? Complacency with non-technical leadership? Lack of integrity and deontological code of conduct enforced by a professional organization of software engineers?
Follow-up
I am gathering ideas from the HN crowd in real time, and it seems that at least for the cookies problem, the way out should be to pass JWT in session cookies instead of Authorization: header. This will solve all problems with binary downloads and reauthentication across single-page apps, especially if the JWT would authorize many roles for the user simultaneously.
