Saturday 14 November 2015

To settle down and feel content

In 1997 I started working professionally as a programmer, to the surprise of my family and friends. I really have a masters degree in economics and sociology, majoring in marketing. This, however, felt boring to me. My real interest was art and computers. My brother bought a ZX Spectrum when I was 11 and we spent a lot of time in front of it, writing basic and inputting game code from magazines like Sinclair User. After the Spectrum, I went with Atari, which also was the computer I wrote my master exam on when I graduated from university. I've often wondered if they could ever read that floppy disc...

So when the web came along, I started creating web pages. Art and computers baked into one! Around 1995, I spent my time doing web pages for the organisation I was really supposed to write a marketing plan for. In 1996 I worked in health care, and instead of doing my job as a controller, I worked on an application in MS Access where the wards could do their own financial monitoring. And then I realised it was time to move on. I started my own web company and 6 months later, I was sucked into the blossoming dotcom industry in Stockholm.

In the beginning, I was fascinated by the new industry. The career path seemed clear. You were supposed to start with front end development, considered lowest in rank, move to back end or the data layer and then advance to project management. The companies were young and the people were all the same age, but already split into two teams. Developers, casually dressed and not trusted to meet clients, and project managers, dressed in suits and handling all the external communication. I was especially fascinated by the fact that the 25-year old women with the same engineer exams as the developer blokes were all project managers. Was it because they were the ones who could talk to clients, because they didn't feel comfortable in the developer group or because of the higher status? Almost 20 years later, I don't know.

I went into the industry because I had a great interest in developing applications that would make life easier for people and organisations. I don't have an exam, I've never studied any technology. So the first 10 years, I always felt insecure. I felt like I didn't really belong, I didn't know the correct terms, I didn't dare to push my views forward because I was always being put down by someone who knew more than I did. And it was easy to put me down, since my self confidence was at rock bottom. Because - I was still a developer. 10 years in the industry and still a developer! What's wrong with me?!

And I fell into that trap. The title trap. Am I really only a developer? Didn't I design and build that CMS? Didn't I design and build that betting sportsbook? Am I not really - an ARCHITECT? At least I should be by now. My employer definitely wanted to sell me as an architect, since it meant more money. And wasn't it really strange, not having the ambition to become anything more than a mere developer? To top it off, being a FEMALE ARCHITECT! Wow, I could probably get my own TV-show! So I went along with it. I went for a Microsoft Professional Architect Certificate. And the years following that was the worst years in my working life.

Why that was, I've tried to dissect. Maybe it was me, with my lousy self confidence, feeling I wasn't living up to the standard I should have. Maybe it was the organisations, having different views on what an architect should do. Maybe it was the younger blokes in some places who made my life hell. Maybe it was my feeling of being hired because WE NEED WOMEN IN TECH and I constantly had to prove I was as good as a man. Anyway, I woke up many Monday mornings and thought about doing something entirely different. Like, maybe, work as a waitress or something... Something I could manage.

The thing is, once you're in that position, it's hard to back out of it. You tell your manager you don't want to have that role. They say they're happy with your work and wants to help you move forward instead of back. And you have that bad feeling that if you back out, you're a failure. Because just saying that "In 3 years time, I still want to be a programmer, only much better" isn't really the level of ambition most companies expect.

I could say it's only me, but when I look at my fellow developer contacts on LinkedIn I see Senior Developers, Solution Architects, Team Leads etc etc. And I wonder, for us as an industry where we developers talk so much about self organising teams, joint responsibility, flat organisations... Where did this need to rise above the rest appear? Why do we feel the need to say that we are SENIOR developers?

For my own part, I've left all of that behind. I'm a developer, in an awesome team, where everyone chips in and I never have the need to lead anything anymore. With that I mean I never have the feeling that tomorrow, when I get in at work, I need to know exactly what to do and how because everyone expects that of me. Mob programming have changed all that around for me. We build awesome things together, I contribute with my knowledge and experience as do every individual in the team. And every day, I learn something new. I go home feeling that it has been a good day, I don't work late when the kids have gone to sleep, and I go to work full of energy the next day. And after 20 years of building web sites, I actually think that this is the best time I've ever had. :)

Maybe one day I'll write about this WOMEN IN TECH-bit.

Wednesday 11 November 2015

Setting up feature branch deploys using Github, Teamcity and Octopus Deploy

Sooner or later, when you have a website in production, the question arises: how to develop new features without deploying partly built features along with bug fixes? Feature toggles might be the way to go. But that can be tricky to implement, and almost always needs code cleanup after the feature is released.

We decided to try and find a way where we could quickly deploy any given feature branch if we needed to. Since we mob program we mainly work directly in the master branch, but sometimes we do bigger features that need to be looked at or tested outside the team.

Really short version - the quirks

  • Custom builds can not trigger another build using the Finish Build Trigger in Teamcity.
  • Variables can be passed from Teamcity to Octopus Deploy only if they are set up as prompted variables in Octopus.
  • The prompted variables in Octopus Deploy have to be set up directly on the project, not in an included variable set.

Deploy process

What we wanted to achieve was being able to trigger a build manually in Teamcity, by stating:
  • Which branch to deploy.
  • Which config transformation file to use (stage, pre-production, production).
  • Which subdomain to deploy to.
This would then create a release and deploy to one of five preconfigured Octopus Deploy projects, which did the rest of the job by setting up the IIS and deploying the Nuget package published by Teamcity. The reason we wanted to have these five preconfigured projects was to have a dashboard available where we could easily see what feature branch was deployed where.

There are of course many other ways to solve feature branch deploys, but we wanted to have the manual step to choose what branch to deploy where and with what config.

Teamcity setup

First step was to set up a new build template in Teamcity. We have multiple projects and use build templates extensively to help us set up new builds quickly. The normal builds are quite basic:

Step 1 - Build

  1. Triggered by Github change to a certain VCS root.
  2. Runs Nuget restore.
  3. Runs Grunt tasks.
  4. Builds .Net sln.
  5. Runs tests.
  6. Copies minified files from grunt step into package.

Step 2 - Deploy

  1. Triggered by finishing build in step 1.
  2. Creates a Nuget spec.
  3. Packages the artifact from step 1 using the Nuget spec.
  4. Publishes the Nuget package to Octopus Nuget feed.
  5. Creates a release in Octopus Deploy and deploys it to stage environment. The Octopus plugin for Teamcity, which is simply a wrapper for octo.exe, works like a charm.
This seemed easy to convert into a feature build. Small changes in how to set up the Github VCS, how to decide which Octopus project to use and how to tell Octopus which config to use.

Set up custom build

To get the parameters in Teamcity that you require for each build, just set them up as required prompts. These can be configured as text fields, select lists etc. For our BranchToBuild we used text field, but for the Subdomain to deploy to and the ConfigTransformFile to use we added select lists.



Now, when we click the Run button for the build, we have to fill in those fields. Sweet!



One thing though! Custom builds in Teamcity can not trigger another build using the Finish Build Trigger. Our solution was to just merge the build and deploy steps into a single build. It took a few hours to discover why the trigger didn't work, and it hurt a bit to get a build consisting of 14 steps, but the show must go on, right?

Get the correct code from Github

In our normal builds we've configured the VCS root to look for changes in the master branch of a certain repo, or pull requests. In the feature branch we of course want the code to be fetched from the branch we input when we trigger the build. Easy thing, just open the VCS root and use the BranchToBuild parameter in the Default branch-field.



Naming of deploy Nuget package

This one we thought a bit about. We decided to go with a "Feature"-package id, instead of naming the packages according to their different application names like we normally do. The reason for this is that Octopus Deploy needs to be set up to look for a certain Nuget package id. By naming all the packages "Feature" we could set up five identical and application agnostic Octopus Deploy projects.

Creating an Octopus Deploy release

We want to deploy our feature to the Octopus Project that corresponds to the subdomain we chose in the first step. The Octopus projects are named Feature F1 to Feature F5 so we just use the parameter Subdomain to set the correct project. As release number we use the parameter DeployBuildNumber which is simply the concatenated build counter, product and branch name, e g '1.0.9-Product-Testbranch'.



When creating a release, Octopus Deploy automatically looks for the Nuget package with the highest release number. This is not what we want here though. Many builds use the same "Feature"-package id and the release number created might not be the highest one, since the build counter will be different for each build. Luckily, the Octopus plugin gives us a possibility to use the command line options of octo.exe. We have two options we need:
  • --packageversion %DeployBuildNumber% tells Octopus to use the specified release of the package instead of the default one.
  • --variable=ConfigTransformationFile:%ConfigTransformationFile% sets the variable ConfigTransformationFile in Octopus to the specified value, which is the config file stated in our input parameters.
When calling octo.exe to create a release, it's possible to send along variables like above. Important to know is that this can only be done if the variable is set up as a prompted variable in Octopus Deploy.

Octopus Deploy

In Octopus, we normally use a rolling deploy, taking the machines out of the load one by one, configuring IIS, setting up firewall rules, deploying, smoke testing and putting them back in the load if everything succeeds. Now, all we want to do is deploy the given "Feature"-package to a certain IIS website using the given transformation file.

We make extensive use of Step templates to create the building bricks for our deploy processes, it's a brilliant feature. Especially in this case where we want to be able to easily add new Feature deploys.

Deploy process setup

Our set up in Octopus Deploy is one project for each available feature deploy subdomain, F1 to F5. This could be changed to just using one and dynamically setting the hostname and IIS to deploy to, but we wanted to have a dashboard in Octopus where we could see what feature is currently deployed where.

Doing it like this is a fairly simple process. The release gets created in the project belonging to the subdomain chosen in Teamcity. The only thing that actually has to be handled dynamically is receiving the ConfigTransformationFile-variable to decide which config should be deployed.

To be able to receive a variable through octo.exe, two things are necessary:
  • The variable has to be set up as a prompted variable.
  • The variable has to be set up directly in the project, it will not work if it's in an included variable set.
Once the variable has been set up, it's ready to use in the Deploy Nuget Package-step of your process.