Wednesday, July 10, 2019

Running Plantuml on AWS Lambda

I couldn't get that to work, well, not entirely.  Anyway, here are the issues that I've come across.

My project just started using DDD and since we have an overseas team, it is a problem as they can't physically participate in the note sticking exercise.  Inspired by (, I’ve decided to build a like a web page to support the eventstorming syntax on top of plantuml.  To be able to work on the diagram together, I've used with `ACE` as the editor which use for a realtime database.  The end result is pretty cool, but integrating that (esp `firebase`) to an angular app induced quite a bit of pain.  

  1. firebase uses a very old version of `grpc` which only supports node 9.x.  
  2. npm modules that have a dependency on the node version is a mess.  they almost never document which version supports which node version.  
  3. firebase rules (you can set indexing and ACL to docs) only provides the basics.  you’ll have to build the whole user management if you want user groups.  
  4. the app used to set `windows.location.href` to update the encoded UML in the URL path.  when enabled authentication, it went into a loop.  
  5. `ng2-ace-editor` didn’t really doc how to load the theme and mode (for syntax highlighting), luckily we have google.

After jumping through a few hoops, now I have a home page listing all documents and can edit them collaboratively.

Then it struck me I could host the app on `AWS`, putting the angular app on s3 and break down the server with API-gateway and lambda function (after all, there’s only 2 API needed).  this decision caused me more pain than firebase…

  1. deploying the angular app on `S3` is very straightforward, however, since angular is using path for routes, the app only works if you navigate from `/` and will get a 404 or 403 if you put the angular route on the browser directly (as it’s trying to find that file on s3).  I read somewhere that we can use CloudFront to map the error page to `index.html`, but I couldn’t get that to work.
  2. API-gateway and lambda have a lot of caveats.  
    1. the lambda is pointed to the zip/jar file on an s3 bucket.  however, every time you’ve updated the zip/jar file, you’ll have to update the lambda with the same URL to the s3 object.  
    2. uploading a large file (10MB) on web console sucks.
    3. when you’re creating a lambda function, you should ignore the designer setting up a trigger to api gateway.  it looks like you can set up the relationship to API gateway there, but it’s a chicken and egg problem.  you’ll need to publish your API before you can configure the trigger here, but you’ll need the lambda setup before you can publish the API.  
    4. `plantuml` use `graphviz` which is not present on the lambda container.  we can set an env variable on the lambda conf to define where to find graphviz.  and packaged it in the zip file.
    5. can’t be jar file as it’ll lose execution permission
    6. maven also needs a bit of a hack to restore the execution permission when moving files with maven-resource-plugin
    7. the zip file is extracted to /var/task on the lambda container.  couldn’t find where it’s documented, I write log messages to `CloudWatch` to find that out.  and yes, logs go to `CloudWatch`.
    8. it’d be close to impossible to rely on a lot of external libraries.  even graphviz supports static linking, there are some libraries that don’t provide the static linking library.  
    9. couldn’t figure out a way to write lambda function with `kotlin` implementing the Handler interface (most example you can find will do that) which provides the API gateway event.  thus, there’s no way to set the response content-type or other HTTP related functions.
    10. don’t forget to define all response code in the method response.  I once forgot 200, and it got a weird error message which definitely didn’t tell you about the missing config.
    11. the API gateway assumes everything is using JSON.  I have an API that takes in the UML text (text/plain) and returns an encoded string (text/plain).  I have to convert the post body into a JSON (using the API gateway’s mapping template) before passing to the lambda function.  
    12. outputting a binary stream is also annoying.  I ended up have to base64 encode the image file, only have the API gateway to decode it right away.  also, there’s a `$util.base64decode` function in the mapping template which certainly doesn’t work.  luckily the option to convert to binary does work.
    13. setting the response content-type is also a bit tricky.  it’s where you define the HTTP response code supported. 
    14. cold start lambda could take anywhere from 5 to 10 seconds.  it’s very slow.  also, each container will only be used a certain time and it’ll be recycled.  i.e. we’ll hit cold start more than necessary.

I've got it sort of working, but a lot of plantuml diagrams doesn't work because of missing libraries.

No comments:

Post a Comment