25 Jul 19

hey good morning well it's rice welcome

to my team this session is called cloud management with Drupal and ansible

and it's about some interesting tools

and an alignment of concerns that we've solved so this is not your problem


because it needs to be a problem that now it's not a problem creating service is really easy actually now because of

all these different tools alright so you

know whether it's Amazon resolution there's a million different ways to get servers now any ones like the credit

card and the web browser can log in and and get a server basically usually you

know depending on where you work you may not be allowed to use these things but that's kind of about the point the point

is the problem is actually configuring and tracking that once you have them is still kind of a challenge because

there's so many options so there are solutions to this in fact there's like amount of solutions though this if you


get into DevOps you may have heard one or two of these things this isn't even everything probably but this is one

hardest thing for me when I started to learn about like servers and configuration is like where what are all these tools like where does one thing

start on the other hand and like you know how do you actually figure out what's going on and so I'm just briefly

zooming in in this grainy photo because there are like recognizable things in here it's like they make a cycle of plan

develop tests release and operate and you know there's tools like Travis and

github and Gawker and it's funny because like docker shows up and and most of these things like it's down here too and release in its

you know so it's it's you're right to be overwhelmed it's a crazy industry right

now so what we've kind of realized and settling settled on for a while now is using Drupal as an actual like dashboard


for creating and managing our other Drupal sites and it turns out the truth

of Drupal distribution that we use has a very very good alignment with the ansible and servers and it's old

distribution called eager it may or may not have heard of the word come from like god of the oceans and so it

commands drops of Drupal sites so people use they go to manage like hundreds or

thousands of purple sites the UI is Drupal itself it's like more than eleven

years old mmm it's God it's you know people love

her hate it but it works and the reason it works is it's all automated like all the configurations are not all the Pachi

Pachi part is office automated right so it doesn't really break most of the time once you know how to use it and it tells

you if it does right so you get these orange or red tasks I got involved using

Hagar many years ago and I wasn't quite good enough for me as a web developer I

kept timing and I customize it and so I created this other thing on top of bigger which is just since it's Drupal those really easy to expand and I made

this thing called open def shop which makes it more of like a developer pipeline like you might see with a great

cloud or Pantheon but again this is just a Drupal site and PHP code templates to

kind of visualize your sites and automatically deploy them and when you

get pushed in them all these run tests and all these things this talk isn't

even really about this part of it of the tool this talk is actually about the service underneath all this there's the

node type actually they call the server and it's stored as information about that server like the host name and the

IP address and what services are running on it and so this is like very this is basically the same problem everyone has

dealing the servers where's my list what's what's supposed to run on it all those things

like almost an intranet for managing your servers you can create a new new servers and notes so mmm actually

getting it working this is the old way doing figure it would be like you know

going in and logging in and you're like how do I install you know true pool and manually getting a server don't even try

to read it so I'm it's just an example right we're not do we don't do that anymore we don't like to do that but basically even even up till now the ager

system expects you to kind of prepare the server by hand to make sure it's ready make sure you can connect to it

and then the web can do the rest so once from a user perspective it was

very challenging as you go to create a server thinking well that's a big promise I'm gonna create the server you

fill in the hostname and then it goes to this and you get a Red Bear five because like there is no actual server there or you know because you're still learning

about how this thing works so this big help text box I actually added to the

depth as a Gurkha had no explanation whatsoever about what this really meant when you were difficult add a server so

we're all working on this open source thing to try to make like a clean better cleaner better or more of automate

experience so there really should be a better way for this right because we we've we can automate the Installer Agra

What is Ansible

itself so we should be able to take it to the next level and let the system configure the server as well so with

ansible is kind of a common language you can configure any server as long as it's not like the same similar operating

system so any of these providers can give you just the raw box and you can reuse danceable to turn those boxes into

any type of server so like free this is a little bit different than like say you

commit to Amazon and then you commit to using their like ELB product which is really just like a proxy load bouncer

and like their RDS is just like that database basically you can get locked into that and then you don't really know

you know you can't necessarily easily move out of that because you think you're you're hard-coded into your PLDs

and databases but advanced ball you can just say - we also stamp out the same

thing every time no matter what it's bringing hosted it even if it's not on a clock you can use it to configure raw

metal sir and the reason this is all possible and easy is this brilliant

Jeff Deerling

person named Jeff Deerling who is super famous basically in drupal world and ansible world now he was actually just

hired by Red Hat which is great news they own ansible now he maintains a ridiculous amount of instable rules that

all work because they're all automatically tested with the weights like Travis CI so they're so stable and

good that you just think they almost always work and he's very responsive to the changes in flora quests so whether

you need to manage like any software basically he's got Java roles of doctor roles in Drupal roles all these

different things to basically almost basically anything the computer can do we could now kind of automate the thing

what these roles are is like just a simple ansel is just simply animal right

and so it just lists all the things needed to do to the server to make sure it can actually run the software all

these things got put together in his pet product pseudo product he created called Triple D yeah you may have heard of it's

been around awhile longer than some with like the local development tools it basically just collates all those roles

that he created and bigger and bigger and it gives you a Drupal VM right and

you can put its makeup sometimes you want it's a standard like a goodie box and it's configured the same way as you

can use these same roles to configure thrush in and configure local and it's like modular right so it's a really

interesting idea we went another level to kind of because our product is like the dashboard kind

of does a lot of the automation stuff so we added a couple more rules and then we're using the hearing guy lamp roles

and extra just very there's like just a few steps in each one of these that to add

on to the existing Apache my spoken things that let us give us the whole dashboard server available to you

so ansible is really just this command line it's a bunch of command line tools


there is this thing called passable tower which is like a UI it's not like

it's you would you have to pay for it for like I might get a license what you reaches like four or five nodes I think

and it's also like it's it's very just kind of like job driven it's not like a

UI for developers right so if you're using ansible you still need to know how to use it there's a number of steps you

kind of have to get familiar with these terms and what a role is and when a playbook is and variables and basically

you kind of step through this and you learn how these things go and you eventually get get the knowledge of how

that work so but even at the end of the day when you're expert you need to store your variables and your inventory

somewhere so you have like either an inventory file which is like a who's file is list all your servers you have a

playbook and the only individual variable is it kind of like you either you are basically either editing the

yellow file on your server somewhere or in source code to change things or you're using like intimate are basically

insult our is an inventory itself it's like dynamic inventory it provides this information to the command line so that

it can configure stuff so basically like it feels great and all but like you

still kind of have to know what you're doing for it to be truly kind of a automated thing and it doesn't solve the

problem of creating service I mean it does solve the problem creating servers but it's a little tricky like you have to commit to one cloud provider for

example like there's a ansible module for easy to and digitalocean to create things with yamo instead of with like a

button and i'll show you in a minute like we created a UI for it where you create a button it's abstracted so this can get a little

tricky as you can see it's like some of the stuff is pretty complicated with it so it's not like a perfect

system Ansel's more about like running and things I find it a little cumbersome

to use it for like creating something or interacting with like another API and so

Cloud Module

what we did is realize that like Drupal is a great framework for building apps

right so he built an app to manage we extended this app called eager that

already had servers and like services and contract' my school or Apache and we

realized those seemed the same way that it organizes the services it's exactly

analogous to sensible so and we can create a cloud module that simply takes

that create server form and actually creates a server and act like teens the cloud EP is these servers server

providers and can actually create the server and then the other models or Hannibal configures it I'm somebody

putting these two together we have like basically a fully automated you create the server check a box telling it what

you want it to be it creep it spins it up because it's done yeah so the UI is

basically this is when you click Add server you can select where you can even

have multiple clouds in your UI so you add your key for digitalocean or a

packet and then when you select the devotion this form appears and it's custom for digitalocean like every

provider has a slightly different way of organizing their stuff so like they call it a region others call it a data center

the image can be called something else as well but basically every API takes

some options right creates a server and gives you backs and information and so

we set it up to just store this stuff abstractly and you can write a simple module basically to extend it to other

cloud providers it's just a very simple class and your pooled PHP nine in Drupal it's just

PHP classes so that this hosting service provider class is a base class where

every single one of them you need like an API all of them have PHP libraries first of

all so it's super easy tray in the next step with the dissolution one looks like but the form function is this basically

gives you the ability to control what you just saw the region selector the image selector the side like the image

size select they're coming gigs of ram or whatever and so we can custom build little forms for each one of these oh

it's provided by the module this hosting service provider as a parent class is yeah it's in the module and then there's

like a subfolder it includes digitalocean packet and software by default in the same repo same hosting

bigger cloud yeah right well it's not

not the features necessarily but like the the interface like the thing that the methods under needs and it's all

defined up there and so the yeah like these these strings are used to create

this like this right so yeah so let's do cookies you can basically make one class

and like have all that information interact with any of the cloudy guys with their options and even destroy so

there's like a destroy method on that digitalocean service class that loads the API calls delete on the droplet and

just sends a message the cloud slash

projects like Edgar underscore cloud and it's just for men remember this is like

an add-on module for the larger like eight your quest master system so there's a distribution you have these and then add this to it so it's not like

a generic we can add to any Drupal module the next I would like the future to be that that

we use down the server module lights in any Drupal site and then add the server cloud module into any site but that's

one other conversation anyway yeah it was like it was really a Eureka moment

it's like this we can make this really analogous and make it really basically just like create update destroyed or service and it's it really works and

it's a lot of fun so the settings page just takes the token I even auto like

digitalocean has a cloud API setting that's record so we can even automate that this just sets the default so when

I go to add a server it's off defaults in New York defaults for gigs and default statement too because they like

you know dozens and dozens of different images and that all works together like

this so when you go to the way it works is you create a server and then there's tasks that run against it so attacked

like this task type so this is a verified task tries to make sure the server's really working it checks the

seeds that can access it and then if it's a lamp server for example it'll check to see if you can read Apache I

try to see that I can access it and create databases and so that's my module

code for either cloud it creates the server and when you it pings the again

when you first submit that server forum but then the verify task is the one that actually finishes the setup so it runs

it waits for the IP address to come back from the cloud API so that I can create a DNS and then I can literally just

click or access it by that URL as soon

as this thing is done it's really this

is like an example of the data it stores if you would want to get into it like so it's it's abstract enough that we can

use just provided options it's what we send the cloud API and data is when we get back it's just serialized so we

don't even bother making like a database table for all this stuff because you don't really need it and yeah it's like


a very heavy it's a lot of maintenance work basically to support more even more than these people one and

other things it's a it's they're all at their own nuances so I could be challenging so we definitely are open to

getting more people involved in the open source side of it AWS for example is

like one of most complicated probably the most complicated cloud hosting provider so I haven't even bothered like

people always ask about that one first it's like well there's so many options is gonna be a challenge so once you can

Drupal Ansible

create the cloud service you need to like configure them right so we the next step is this ansible thing so the cloud

provider dad I was handled but now we needed like the it really I mean that this is a perfect system to manage your

servers with ansible is just like it's so it's similar ansible tower and it's crazy so we have server nodes so we can

generate an inventory lists of your servers we it already stores in the database like what server runs which

service of you've got roles we've got variables even when you go to add the server you tell a career like what the

username and password should be so it actually already stores some variables we just have to basically I just all I

had to do was reformat them and print them out in the format that the healing guy role expects so my school user home

in root username and password I the data was already there sitting

there and either in the database and at eleven-year-old Drupal distribution so

this is really fun to me just a so we just loaded it up and spit it out and ansible readable format so there's

already a task queue integrity much you know so I'm you've gotta got that lock that stuff this is basically analogous

to play books play book runs all the stuff we needed is right there you got

this is what a verified server test looks like when it's actually running the ansible and we've got like

failed state so we've got already you've got the timing data in there it's

like none of this had to be it was already there all I had to do is hook into the verify process to run the

PlayBook run even the logging was already there there's already a posting log stable to save all these rows and then I can spit them out to the user and

yeah it's all a gamble so it really it was amazing how little time I mean it's

this stuff's actually coming years old but it's a great little time to kind of actually get it working if you get into

ansible deeply there is a system called the dynamic inventory and so this is how

people use you can basically there's a plug-in where you can say I'm using Amazon so my inventory is all of my

Amazon servers and so if you install the right plug-in when you call ansible playbook it'll automatically run it

against all of your ansible servers using the inventory and so what this does is provides that list from a group

aside so that way any server anywhere because it's a website can pull that list dynamically so I can create a

remote server somewhere hook it up to this inventory resource and run ansible

from anywhere and it'll pull in all the variables for that server itself then config and be able to configure itself

so that's really interesting too so yeah like I said we used ansel fears Jeff's

roles have been in there for years we made the role generic so that we can fire up new Apache and MySQL servers and

then the automated stuff then layer on top of that and it was like well yeah

what have we tying them together and that's what we did so

yeah so this is what you would do when you're going to create a new server in Hagar there would not be that ansible -

cool option you would actually so that he saw that form a second ago you'd

actually have to config set the server set the root password and then come back in here and put it in so that the site

could nut could access it and automate the database stuff we don't need that

the danceable because it's ansible so we just configure that we make a make make up a password for you and serve that as

the variable to them to the role so it was super simple mmm - just like hide

the form for passwords and like generate it save it it was already saving it actually bigger saves it in there and

the same thing with Apache the existing Hagar you would actually have a feel for

what's the Apache restart command like instantly and actually say that and use it but since we know what it is already

because you're choosing your configure an advancable and we know it's like fixed you know it's a fixed operating

system we can get rid of that configuration and just allow people to mess with ports if they want to in say

so then we kind of were like - took the next next natural step we made roles


standardized for Apache as well so we can install Drupal but clearly this can be anything because there's roles for

everything so and with the ansible galaxy it's as easy to download a role as it is like you dude rush to yell you

know Drupal you download a module you can do insel galaxy download or whatever they install as the command and then it

pulls that just by name and it'll pull that role down automatically and you can use it instantly and so we realized oh

let's make our own UI for that and you can add roles to the website say these are the rebel roles you just type in the

name you can override the git source if you have a fork or something and it'll download those roles onto the server so

that are available and they're here so that when you go to create a server you can just check those boxes for each role you

want so this could be anything there's roles for Jenkins memcache

I need software and you can make your own if you needs it right so it's it's really great because you can you know

managing the stuff with like editing any mo file is challenging so this way it

gives you an actual like UI way that kind of turn your it's a server SMS CMS

and then variables was the next logical step like we provide we can provide a


hard-coded variables to make sure it works the way we want it to but the ansible variables are just the Emmel and

the animal symphony saml parser and all that so we made a text field we put in any Hansel reimu you want and it merges

that into the inventory now way you go so you can literally go in on the server

type in a memory limit change it save it

does actually validate the mo thing in front ends late it'll tell you you put something in wrong and then that

variable is merged in with all the others that are some of them are automatically generated as you can see here and it allows you to basically

gives you a web UI to configure your servers the variables are like predefined things so like your in

diagram at M plet that injects the MySQL all these - ago things right into the right place so you can basically look up

the ramiz on all these roles and see what the player goals are available to you to tweak your server without having

to like manually edit a config file and yeah this is the dynamic inventory I was

Dynamic Inventory

mentioning for like any anything can be you can write your own ansible inventory

as long as it returns this JSON inquiry instable itself can read it and so this

kind of solves the problem of having like storing your inventory in a text file or having it in a central place

where you can have one source of truth for like what all your saviors are and so by using this hero that we were like

well that's very simple we can print JSON easily and we wrote a little shell script to just pull it down and serve it

to the hexa bol command so when you're on ansible it actually love this data and uses it as a list of things to run

and you do that with a trick with that post I'll get into questions if

anybody's interested but this allows you to basically like not worry about your command line can be very simple so you

can call like what the inventory it's set up dynamically you don't have to use the inventory option on the command line because it's always there it's configure

server wide so you can call you can basically run against all your servers simple commands like this like ansible

all the AMEX for module writing the command and the coulomb is at an actual shell command that they run on the

server but it'll run against all this is gonna run against all servers or you can type in a filter to run it against an

individual's or certain groups and so these were ssh accessible and these were

not and so this tells you very clearly like what's going on so it's kind of

neat because we are able to provide this web UI that you also have access to the

command line stuff so you can still you know muck around with there and still it's not like a black box or whatever it

where it's it's all in one place and yeah it's all just know it's Drupal

nodes so you can use fruitful modules and hooks to alter the ansible inventory for example if you're the guy has a

security role where you can just set things like permit route log into

and it will automatically do that Sh config in the right place for you but

it's up to you to set these variables and do these things so either you're using ansible you're either editing your

inventory 500 and your bars and making sure all those things are in there or we were a triple module so that every

single new server we create gets these variables and gets these roles at it as well so every single circuit we create

automatically gets the auto updates oh don't have updates are configured is sent to me I don't ever have to think

about it again because it's dynamically loaded into the inventory of every server we create and so that's cool

because we don't remember to check the boxes when we go to the web UI sometimes like oh great gay people flexibility web

UI but now they don't know what to do sometimes your system you want to hard-code things like that brings

another thing we do for example is there's to the security role is cool

because there's a sudo password lists variable that you just provide a list of

usernames and it automatically creates users on that server for those names and it automatically adds in the pseudo

group and allows them to run commands without having to reach enter password at all and then there's a second roll

called github users which creates I

think it creates the users as well but it automatically pulls the SSH keys from

github for that username and puts them on the server so that you can access it from any of the computers you use

personally so when I go to create a server all of my get my keys are always

put on there with the user under my name and I can see them so every new server I

get I can just instantly as a say chain is jumping it's like

while I have my private keys it turns different you're going to management is

what I'm what I'm saying alright so if you go too you might not know this but if you go to slash get up icon slash your name dot keys it gives you all

these because it's public those are public keys and it strips them of like the identifiers so it's a really easy

way to like pull all your public keys for one person right and that cuz they commit we've all maybe as many of us

have multiple laptops and other things it's much easier than saying like send me your public key because then the

second thing goes their computer it's like I've got to do it again alright but if it's this way github has all your

keys all the time so it's like it just it's automatic it's really really cool and I'm just the messenger

Jeff stuff but the point was we were a simple module that just alters the array

of the node and Hartman prints out the the people that I trust to be my sis

admins basically into both of these little variables and then every single

time I create a server I get both of those all those users find this is--these hisses readme so you can even

do more than I'm doing you can actually specify what groups they go into and all this stuff and more most importantly who

to remove so like when people have you gotta remove people it won't automatically remove from a list but it

will have you added to that absent list next I'm going to make sure it takes that person I like this they get fired

Ansible Playbook

the final toolkit we made was just ansible playbook it's just a command you can write on the command line so we were

like and Edgar has a task note that you can add arbitrary parameters to so

that's when we did we've created a playbook task and we add parameters for all the different options in the actual ensel playbook commands so you can just

type in an actual raw pack to a playbook file path to inventory the limit is like

the string that you filter by so it's all or weather or whatever the user even so

this is like totally custom basically just a playbook command with any of these options and it just runs it boom so we

did die a girl tries to show you exactly that's running so you can copy and paste that command if you really want it to

and run it to debug it we think this is

like really important for any kind of really real stable IT cell hosting thing is you know you can configure a server

on the command line a million times but are you blogging that like who ran it like the output just goes away and you

know forever basically you know if you're running it manually in the terminal but this way it saves that past

node basically forever until you delete it so you can go back and see like what ran what was the output who ran it even

because it's a triple note so the author is in there it's it's a much kind of

more more sane way to run things because you actually get visibility into the server without with getting some history

so again this is like I'm gonna kind of at the end of my slides so really early

I had a trying to run a business but it's all in the spirit of like these Loy's roles are open source and we it's

hard to maintain them so check out what we're doing if you just want to play with it we provide community support in

the chat rooms always because it's all open source just like a Drupal module but we also will back it up with like a

support contract if you really want to get into the nitty-gritty and it's fun because our clients are a lot more like

partners like they runs around data centers that run their own servers in some way or they have special networking

requirements and they also know how things work and so they really like the idea of like finally automating what

they already know Apache servers they know all that stuff they're just trying to like figure out how to take some next-level

so yeah that's the main bulk of my talk

and if there's still time I could do some demos questions for the get part

[Music] are you talking about like the Drupal

yeah yeah yeah that's actually part of like the Deaf shop workflow so yeah

Drupal UI

that's exactly what it does so for the screenshot here did tab oh I see right

this is just the settings page this is a screenshot of the settings page yeah let

me just show so the the web UI for you

there's a wet there's a section that's for your Drupal projects and as a section that's for the service so like

this is our this is our main website and then we have different environments so

live is on master and every time you create a new environment it uses the main git repository for that so I can

just click create and you can change you

can change the branches this is to create and if you want to say like this

this is the Thursday environment I guess I created on Thursday two months ago you can deploy code and choose a different

branch to deploy it on so every environment is easy to this is like a

manual in process but we also do like don't automatically to create a pull request environment things like that

yeah you know so you check updates and that's like just play it the pointing updates one and it'll run all the you

know database updates and stuff like that but yeah mostly this talk is about

it's this so while you yeah did that answer your question I mean I would say

not sometimes yeah so it's we take a

slightly different approach in that like you actually need to do very little to get the site up so if your CI is like

building a whole server from scratch even if it's a container like you're doing more work and you have to do right

and so this is a single server and it just clones the code composer installs it and puts it database in place and

configures a file so it's actually like a lot faster than some of the other like like more darker solutions and things

like that because it's just the wrong basics right so I can just type in a name choose the branch choose what I

want to install with Drupal give you options try you want to install anything I'll just clone in our live site actually and their settings but you can

ignore them and then you just click create and or not like there's a lot of occurrences a lot of benefits that these

like native services is just the lamp stack also like composer installs run on as the same user every single time so it

uses the actual composer cache and every composer install is very fast you know a

30-ton like that was 11 second composer install for this many packages and if

you were running that in a container on some other like if you lose all the cash that's doing a fresh composer soft every

time right so it's a lot of I get I do a lot of consulting for CI projects and it's like how do we speed up the build

it's like actually simplifying the system why should speak your bills yeah so anyway that's all

though that's the stuff I was not gonna show but yeah this - server stuff is really fun what's going on a server for Def Con so you basically just spikes

Server Monitoring

like actually this is gonna be out of date I have to update this list so I can go to my settings for cloud this list

changes so you have to like click this button refresh what's interesting about

this too actually is the image can also include your backed up droplets so you

can kind of spawn your own on the old servers back up again so if you really wanted to create a system to like

archive the server spin it down and put it on there you know put it on their

thing you can

there's sure that testing Venus we have server

such a listen the defaults are fine these are SSH keys are even stored this

is digital ocean speed I've added a new digital issue but I don't even need that because it's gonna create the I'll show

you this is a custom role like so we you can do this like one at a time thing but we're posting at clouds

you know we're hosting service we want every server to have to be the same we want every server to have all of these all the time and so instead of forcing

ourselves to check that we made our own little custom role thing that simply you check dev master you put our license

keen on the version and that's it and so also show you on the other end I all these variables are populated for us by

our little custom module that I showed you with as like the security variables and all the things so this is already

coming back in digitalocean and there's the variables that we generate automatically from our module

so all is you can the chin up for example and so that way like I don't have to set up the monitoring it sets

itself up automatically and there goes so it's waiting for the dns to be active

and it waits for the ssh to login if the drop is the brand-new droplet some

things sometimes that's ish doesn't quite let you in right away and we have

to restart it but sometimes it just works the first time but point is we kind of we're designing it handle

Ollie's a little these little situations

sometimes it just won't ever we'll just hang on that one thing but it's cool it's a hit cancel

I can restock but there goes so it generates the playbook based on the information tells you what it does we

should actually literally dynamically generate this and then run the PlayBook file against our inventory and there

goes and it's kind of funny actually it happened so fast the apt-get lock hasn't

been released from the initial server creation so I haven't had it a wait

period for that yet so I can just hit retry no usually works a second from


logs or Watling locks yeah nine different ways like I right now I'm

using a Ginga you know so it we pretty much don't dictate that I guess like the standard Linux logging is happening

so like var log is full of stuff we're with Apache and then we will you know we

write out a different log for each site for example and then kind of let people go from there because if the log files

are there and people have different opinions what service they want to use yeah yeah exactly so that's what's cool

about the answer there's ansible goals for log stash all these different things and basically because it's a simple in

exact we don't have to take long when people can kind of choose diplomat really has their own desired service or

whatever whether it's Ginga or you know there's a million other log services now

but since it's just using standard logging practices on Linux it's relatively easy to to do it so there you

go but it tells me it was then couldn't hit the chinga server for some reason but I

gotta do is roll any other questions

I'm not quite yet but the the using an slike this is where it gets kind of

tricky and fuzzy like I found it tricky to use ansible for dating service and I found absolutely tricky for doing the

actual deployment sometimes like I have a separate command that basically runs

like the deployed like that get pull and like cache clearing but basically that

is a feature goal we absolutely want to have and people they automated but it's

not quite there yet but yes so Pegasus I think my anger might be down or something oh no I mean the very first

one even for one yeah I mean it our install script is basically ansible were

like it's just running ansible goals so I mean it's fun to do it by hand as you

to learn its know what's going on but like that very first time you get back

the server and you're trying to be created again over here and you're like what was that one thing I did manually

that I forgot to write down you know what it means so it's just kind of best practice to like go ahead because I mean

if you're doing it you can always do just like apt-get install lamp if you really are comfortable with it you know

but eventually like it won't be long until you need another you know another step or you you want to do it it's very

like please you know once it's if it's all pre-configured it takes so much less

time and so much less mental effort that I mean if you have the time to do it great you know go and that actually if

you're doing that are knowledgeable enough about it you can help contribute to the roles themselves actually because it's still like oh you know it's kind of

an open source effort like up until now Jeff did all those things basically just in his free time and also as part of

this Javadoc weeow because they they actually do do like custom deployments for certain clients

not just podcasting but now he's actually paid by a Red Hat so there's going to be some interesting

things happening but yeah it's not just for Matt I don't think it's for mass

hosting just at all I think it's just it's it's so much the same it's just everything standard insane I think like

editing I'm editing a config file on the server on the fly is just there's no like it's but it's challenging to figure

out how things got that way or whatever and you can spend anybody on the terminal to spent many hours of the many

days figuring out kicking machine why is it configured wrong so by using ansible it's all just the same and the roles are

tested meaning every time he pushes a commit to any change of those roles it installs it in like four or five

different operating systems on Travis you know so it's actually tested in multiple operating systems and it's

ansible provides really good tools to do that to make automated testing easy and

I've actually learned a ton about Linux itself just for reading rolls and reading because it's the animal it's not

hard to read you can go in there and kind of figure out what it's doing and

but yeah it's if you're not on to me is some degree like it's gonna be more work

behave it some people's Nestico depends on your situation yeah yeah and at the end of

the day sometimes ansible can't confuse you and you might not know why something's family so you do often times go in you know poke around manually to

figure out some strange behavior

[Music] you have questions

Yeah right now it's all it's all that you know built on top of this agar stack

but the absolutely see a benefit of making a new version it's not eager it's

very abstract for servers and services for rupal because a lot of people big

companies big hosting companies even use Drupal grow-ops and used it for office management in different ways there's

actually I should show this there's actually a really the cloud module is now a thing again it's one it's a very

old project on Drupal at work but it's now maintained by this massive consulting company don't como me just

Japan and but it's like they built the Amazon they built it so they have like

everything Amazon oriented already in a Drupal UI blade and it's all I've

tracked anyway so yes it is Amazon but all the tools they built are supposedly attracted so this can be extended to

become the next generation who knows what

I think that's should be the end of our time right next session starts attending here there's no more questions I'm on

the board for a lunchtime tomorrow you like bottom or Indian Thanks


The Aegir Hosting System has been used for hosting thousands of Drupal sites for over 11 years, using Drupal as a web interface for managing your servers and sites.

In classic Aegir 3.x and earlier, you still need root shell access to install and configure a few things before Aegir can work.

in 2016 I set out to solve this by creating server configuration tools in Ansible and integrating with Cloud server providers like DigitalOcean, Packet, and SoftLayer.

The result was the Aegir Cloud and Aegir Ansible modules.

Now with a single form from node/add/server, you can:

  1. Create a cloud server instance with your choice of data center, OS, memory, etc. and automatically authorizing the Aegir user's SSH key. 
  2. Select the services you want installed, like Apache or MySQL preconfigured to work with Aegir.
  3. Add custom Ansible playbooks to each server.
  4. Add custom Ansible variables as YML into a simple text field. 
  5. Automatically discover the server's IP address and set DNS records for the server's hostname.
  6. Wait for SSH access via root.
  7. Run the chosen Ansible playbooks with the generated and manually entered Ansible variables.
  8. Get a Red or Green or Orange result if any of those steps failed.

Come to this session to see how we are using this combination of totally free and open source tools to power our platform-as-a-service,, where we create and destroy DevShop servers on the fly.


Slides available: