Recoupled Drupal for VA.gov

When i came in it was really um the main challenge was don't break the build.

There was the decoupled process, was very decoupled right, as kevin said, the web project existed well before the drupal came along and so they treated the drupal content really as just another data source.

So they weren't fully ready to commit to drupal as like source of truth they had all these other content sources that they were also compiling into their build and so what ended up happening was the web had their own devtest live process they had their own prs on a different system but devtest and live inside their build release process ran the full a full kind of web build.

It pulls content at that time from cms as well as the markdown and all the other sources that you were talking about but that happened like at exact build time and it also happened from dev to dev, test to test, live to live, so there was a web build stack there's a web stack with dev test live and a drupal stack with dev test live completely separate and so there was really no telling essentially the cms would just get hit and pulled the content and so there was all this instability because the drupal site was being developed quickly as well.

It was not only that but the devops was kind of being rolled out at as we went so there was va.gov requirements, aws gov cloud requirements that were causing separated it from they were the two servers were separated by a network they needed a proxy uh meaning the web builder and the drupal cms sometimes the site wouldn't respond because of, you know, any number of reasons and then

Sometimes it was graphql specifically, sometimes the graphql worked but it crashed with memory errors because the content was too much. Sometimes it errored because of a schema change. Sometimes it errored for any number of other reasons, the problem was the the web build process was happening on a completely separate jenkins server so the drupal team had no idea what errors were actually happening at the time.

And the web team effectively since the devops for the drupal site was still early and all of these technical issues were making it look like to them that the site was not reliable so they coded around it they were a totally separate team. So they actually coded into the javascript build process it like this just completely unbreakable build, uh the drupal site was an environment url, it would try to load the content from there.

If it wasn't available on the network it would use old content. If it errored it would use old content. If the graphql errored it would use old

content. If there was a bad schema, just a simple field change, it would use old content.

All this was happening without kind of the cms's knowledge, and so we actually would it was a consistent problem that the public production deployment would actually send out old content.

It wasn't until we unified these build processes that we finally figured out why that was even happening. It was even hard to figure out we learned more and more about the web build by doing this and learned that it was we didn't even know how much caching it was doing until we started to unify this process. 

So what would we do, um we said "hey. you're you're a javascript project but we have this really great dependency management tool called composer", right, so we got them to actually submit a composer json file for va.gov web so that we on drupal side could do composer require va gov web. 

What that means is that we could actually pin the exact version of that decoupled site,  that website's repo, install it along with the composer install process and then test the whole thing. Then we could even send out test results based on the commit, so for the first time we could actually have the web code and the cms code running in the exact same spot, against the same exact commits, so we could reliably kind of test code to code.

An interesting side effect of this was to create this node project type, so I don't know, I'm kind of thinking about this as a model for decoupled sites because you're always needing to develop two things at once. It's a very common use case to get repos, it's there's never really alignment so we're thinking that because the decoupled space is getting more common, we can have like a node project and it knows where to put it. It puts it in the slash web folder here. 

Other tools that we needed to solve this puzzle was the composer install post install command, to save time and also just be able to make it automated, we hooked into composer installs itself so when you run composer install, it's doing a lot of drupal stuff and git hooks but down here it kicks off to this va-web-install command. 

This is the exact same code that the npm project uses so we basically emulated the web build process inside our composer install process this is the npm, this is the yarn install, uh proj uh command i forgot to scroll this over because there's a it's a very it's totally similar node.js and and php. 

You've got the composer install process, which just deals with the code, the source code gets scaffolds, all the dependencies, npm install. yarn is used often that's just the source code but then we go and we get also build which composer doesn't have like a standard way to do this but in drupal you would be call it the install process, or the build process. 

The va-web-build command we created assumes there is an actual drupal site available. It actually this is pretty cool once we have this in process we were able to wrap it with some extra things, like for example how do you tell the content editors that there's a build happening. A config set so we created a little thing that sets a variable so that a message can be shown in the front end to the developers or content editors rather saying "hey your content is being built uh in this backend build process" um it does a little cleanup it does extra stuff like we couldn't get around getting these assets um we needed extra content etc the yarn build process needed this node environment variable but we were very easily able to just add that here instead of worrying about you know environments and dockers and all these other things we could just drop it right into composer as a variable to try to get the thing to work to get to make sure that no matter what you did composer va web build would actually build the entire front end from the site you're running it within like 

That was one of the biggest things, was there was no like site-centric build process so this command always looked and hit against whether it's the local development environment or the ci environment or production and so we really started to get uh t's out exactly what we needed and then at the end rush config it's done. 

So this is just a composer in the composer scripts list it's just anything you add here creates a composer command and so it's a very helpful way to this is core what devops is all about to me is giving the developers control over these operations so highly recommend to kind of like use it to bridge the gap between npm and composer.

Then the final the final big solution of the puzzle was actually getting the environments the the ci sites to do this right. Because without PR um without putting the web in the CI, in the PR environments there was no testing. Drupal PR site would fire up on a branch, but it would just use the main branch, you know, use the main web. 

It would actually, i think at first, they were all pinging, they didn't reach out to the web build process at all. The web dev site simply reached the CMS dev site, right, and so there was no automated way to build the front end, and test it. So what ended up happening was the schema changes didn't even get read by web build until they were merge. 

It was like they were immersion put to dev and then we found out oh the web build wasn't able to consume the new content, and so to solve this we were able to by with the composer install and the composer va-web-build just became another update hook.

We unified our CI by linking in web. So that every single git push does a new composer install, which kicks off the front end builds install, and then when the database comes in, and it tests the deployment part, which is like your update database cache clear, it does another web build. 

So this way we had like full CI for every single step of the process end to end, and it was it wasn't until that point that we really finally even could start to identify why there were all these incompatibilities and why the the build was unstable.

So real quick some some interesting solutions were coding the web build script to output to just simply static.  It turns out every drupal server can also host a node project if it's just html, so we dumped all the static html right into /static and no matter what tool we're using, lando, devshop, drupal vm and then even in production it would all go render static and we didn't need to worry about a node container or any of this other stuff. It's just because it's hosted by apache servers html perfectly well. 

So we were able to test in any environment we could just go to static and boom there was the the associated front end we did an apache config trick which allowed not to /static doesn't always work for you know your relative routes and stuff. 

So we actually did figured out the exact apache config to, with a little bit of regex to create a second virtual host for every CI site um and so we had pr123 and then pr123.web. 

That way we knew we had a predictable url pattern for one site but two urls and then,m one last thing that um I was able to add was i wanted the use case was, if someone's editing a node or editing a blog or in inside the site somewhere, how do they get to the static version of that page? 

So i kind of came up with a simple REST format class. It's in custom code right now but this might be useful for contrib. 

All it does is it it's the same thing that you see for like format REST or format i mean sorry format XML or format JSON, format "static html". All it does is look up the if there is an index file and loads it and returns it. 

So this way it kind of unified the drupal site itself. So we had like the node, the drupal admin ui page, because the entire thing was admin we should mention, i don't think that was mentioned yet the entire cms is backend there's no theme at all it's all just drupal admin pages. but the url's always matched so they did use since they use the urls for the content both in the cms and the front end you could jump from back back and forth with each other with this little format static html thing.

I think the key takeaways here would just like, make it all, you know you got to get these things together or you're going to end up with instabilities and uh that's it pretty much it.