Thanks to a rockstar team of Ubuntu developers, generally speaking I don’t have to care about systemd as whole. I do have to care about the user exposed surfaces though. Here’s a good way to get started:

Thanks to whoever put this together, this is a brilliant way to get people up to speed.

Check out how Project Calico is using Juju.

Installing OpenStack is non-trivial. You need to install a large number of packages across a number of machines, get all the configuration synchronized so that all those components can talk to each other, and then hope you didn’t make a typo or other form of error that leads to a tricky-to-debug misbehaviour in OpenStack.

And here’s the bundle with the nitty gritty.

On a side note I found this page interesting for those unfamiliar with Calico.

This title is kind of a misnomer, as of course, all this goodness is available to Ubuntu 14.04 users, so it’s more of a “Things that happen to line up with” Ubuntu 14.10.

More new items next week, hint hint!

Greetings folks,

The Juju Ecosystem team at Canonical (joined remotely by community members) recently had a developer sprint in beautiful Dillon, Colorado to Get Things Done(™).

Here are the highlights:

Automated Charm Testing

Tim Van Steenburgh and Marco Ceppi made a ton of progress with automated charm testing, here’s the prelim state-of-the-world:

Jenkins Jobs Fired off: 22

This enabled us to dedicate hours of block time of getting as many of those red charms to green as possible. The priority for our team over the next few weeks will be fixing these charms, and of course, gating new charms via this method, as well as kicking back broken charms to personal namespaces.

Ben Saller helped out by prototyping “Dockerizing” charm testing so that developers can test their charms in a fully containerized way. This will help CI by giving us isolation, density, and reliability.

Charm Tests are now launched from review queue to help gating based on tests passing.

Thanks to Aaron Bentley for supporting our efforts here!

Review Queue

The Charmers (Marco Ceppi, Charles Butler, and Matt Bruzek) dedicated time to getting through reviews. The whole team spent time creating fixes for the automated test results mentioned above. We’re in great shape to driving this down and not ever letting it get out control again thanks to our new team review guidelines: http://review.juju.solutions/

The goal was to help submitters and reviews know the where they are at in a review, and next steps needed.

Here are the numbers:

  • Reviews Performed: 189
  • Commits: 228
  • Charms Promulgated: 10
  • Charms Unpromulgated: 7
  • Lines of Code touched: 34109 (artificially high due to SVG icons, heh)
  • Reviews Submitted: 84
  • Energy Drinks: 80

Some new features:

  • Users can now log in with Ubuntu SSO and see what reviews they have submitted, and reviewed
  • Ability to query the review system and search/filter reviews based on several metrics (http://review.juju.solutions/search)
  • Ability for charmers to fire off an automated test of a charm on demand right from the queue. When an MP is done against a charm, we’ll now automatically reply to the MP with a link to the test results. \o/
  • You can now “lock” a review when you’re doing one so that the rest of the community can see when a review is claimed so we don’t duplicate work. (Essential for mass reviews!)
  • Queues divided and separated to highlight priority items and items for different teams

CloudFoundry

  • Improving the downloader/packaging story so it’s more reusable
  • Cory Johns developed a pattern for charm helpers for CloudFoundry; the CF sub-team feels this will be a useful pattern for other charmers. They’re calling it the “charm services framework”, expect to hear more from them in the future.
  • We were able to replicate the Juju/Rails Framework deployment of an application and compare doing the same thing on CF: https://plus.google.com/117270619435440230164/posts/gHgB6k5f7Fv
  • Whit concentrated on tracking changes to Pivotal’s build procedures.

Charm Developer Workflow

This involves two things:

“The first 30 minutes of Juju”

This primarily involved finding and fixing issues with our user and developer workflow. This included doing some initial work on what we’re calling “Landing Pages”, which will be topic based landing pages for different places where people can use Juju, so for example, a “Big Data” page with specific solutions for that field. We expect to have these for a bunch of different fields of study.

We have identified the following 5 charms as “flagbearer””: Rails (in progress), elasticsearch, postgresql, meteor, and chamilo. We consider these charms to be excellent examples of quality, integration with other tools, and usage of charm tools. We will be modifying the documentation to highlight these charms as reference points. All these charms have tests now, though some might not have landed yet.

Better tools for Charm Authors:

Ben, Tim, and Whit have a prototype of a fully Dockerized developer environment that contains all of the local developer tools and all of the flagbearer charms. The intention is to also provide a fully bootstrapped local provider. The goal is “test anything in 30 seconds in a container”.

In addition to this, Adam Israel tackled some of our Vagrant development stories, that will allow us to provide better Vagrant developer workflow, thanks to Ben Howard and his team for helping us get these features in our boxes.

We expect both the Docker-based and Vagrant-based approaches to eventually converge. Having both now gives us a nice “spread” to cover developers on multiple operating systems with tools they’re familiar with.

Big Data

Amir/Chuck worked on the following things:

  • Upgrading the ELK stack for Trusty
  • Planning out new Landing Pages focused on the Big Data story
  • Bringing up existing Big Data (Hortonworks) Stack to Charm Store standards for Trusty, and getting those charms merged
  • Pre-planning for next phase of Big Data Workloads (MapR, Apache distributions)

Other

  • General familiarity training with MAAS, OpenStack on OBs and NUCs.
  • Very fast firehose drinking for new team members, Adam Israel, Randall Ross, and Kevin Monroe have joined the team.
  • Special Thanks to Jose Antonio Rey, Sebas, and Josh Strobl, for joining us to help get reviews and fixes in the store and documentation.
  • We have a new team blog at: http://juju-solutions.github.io/ (Beta, thanks Whit.)
  • Most of the topics here had corresponding fixes/updates to the Juju documentation.

ElasticSearch is one of those tools that’s just handy to have. When combined with Logstash and Kibana, you can use the “ELK” stack for all sorts of things.

One thing we’d like to see is to bring these capabilities to our users. So I’ve been working on bundling together some of our ElasticSearch resources in Ubuntu so we can bring them to the cloud, it looks like this:

This is a standalone ElasticSearch cluster. I’ve added a few things:

A Production-ready Stack

One of the nice things about this bundle is it uses our ElasticSearch charm. Right off the bat you’re getting a charm that we’re using in production today. That means it’s tested (see the included tests, and it also uses many of the modern techniques for writing a charm. In this case, Michael Nelson leveraged Ansible to do most of the heavy lifting. Our ability to consume multiple tools to get the job done is one of the nice things that we can support in charms, so if you prefer a certain tool, then by all means use it! It also just consumes upstream packages, so you’ll always have a fresh version.

Why ElasticSearch?

The major use of ElasticSearch is within Ubuntu’s Software Store for Unity 8. On Ubuntu Touch anytime a user is searching for and navigating the store they’re using ElasticSearch. ElasticSearch was chosen for a few reasons.

  • It allowed us to design the store where every category is basically a search term. This allows the team to dynamically generate categories based on certain criteria. For example a list of “Top 10 apps” might be different depending whether you are in the US or in China.
  • Since everything is designed around search, it lets the team “future proof” the Software Store for future categories and queries.
  • ElasticSearch was designed to scale much more than other solutions we looked at. ElasticSearch can be horizontally scaled by just juju add-unit with no extra configuration.
  • Since the team didn’t have to worry about learning how to make search it let them concentrate on the store itself and let ElasticSearch do the heavy lifting.
  • Since ElasticSearch is driving the backend, this will enable Ubuntu Phone partners to offer customized views on top of the existing store without having to do major engineering work around building a branded store.
  • Our team found the ElasticSearch documentation to be excellent.

The quick way to get up and running

Ok so let’s deploy this badboy. Assuming you’re brand new:

sudo apt-get install juju juju-quickstart
juju quickstart bundle:elasticsearch/cluster

At this point follow the ncurses menus to pick which cloud you want to deploy to and add your credentials and then Juju will bootstrap and start deploying ES.

Total deployment time will depend on where you’re deploying to, but on AWS US East 1 I can go from zero to cluster in about 10 minutes. By default I give you one Kibana node and one ElasticSearch node. For ElasticSearch we ensure you’re node has at least 4 CPU cores and 16GB of RAM. You can follow the instructions to load up the sample data and the other sample plugins I’ve included. Just go to the ip address of the Kibana unit in your browser and start searching:

Now you’re ready to horizontally scale:

 juju add-unit elasticsearch

To add a new node.

And that’s it

You now have an ElasticSearch cluster. Using the same best practices that we’re doing in production. Amir Sanjar and I are prototyping bundling the ElasticSearch Hadoop plugin in a bundle as well, though that’s not quite ready (testing and help wanted!).

As you can see, we’ve barely scratched the surface. Chuck Butler’s been reworking the Logstash charm for Trusty, we expect that to land soon. We plan to make the charm a subordinate, you can just plop it onto any unit. The idea is to enable people to put logstash’s indexer on every unit they can shove all they care about into ElasticSearch.

Prentice Hall has just released the 8th Ed. of “The Official Ubuntu Book”, authored by Matthew Helmke and Elizabeth K. Joseph with José Antonio Rey, Philip Ballew and Benjamin Mako Hill.

This is the book’s first update in 2 years and as the authors state in their Preface, “…a large part of this book has been rewritten—not because the earlier editions were bad, but because so much has happened since the previous edition was published. This book chronicles the major changes that affect typical users and will help anyone learn the foundations, the history, and how to harness the potential of the free software in Ubuntu.”

As with prior editions, publisher Prentice Hall has kindly offered to ship approved LoCo teams each (1) free copy of this new edition. To keep this as simple as possible, you can request your book by following these steps. The team contact shown on our LoCo Team List (and only the team contact) should send an email to Heather Fox at heather.fox@pearson.com and include the following details:

  • Your full name
  • Which team you are from
  • If your team resides within North America, please provide: Your complete street address (the book will ship by UPS)
  • If your team resides outside North America, you will first be emailed a voucher code to download the complete eBook bundle from the publisher site, InformIT, which includes the ePub/mobi/pdf files.

If your team does reside outside North America and you wish to be considered for a print copy, please provide:

Your complete street address, region, country AND IMPORTANT: Your phone number, including country and area code. (Pearson will make its best effort to arrange shipment through its nearest corporate office.)

A few notes:

  • Only approved teams are eligible for a free copy of the book.
  • Only the team contact for each team can make the request for the book.
  • There is a limit of (1) copy of each book per approved team.
  • Prentice Hall will cover postage, but not any import tax or other shipping fees.
  • When you have the books, it is up to you what you do with them. We recommend you share them between members of the team. LoCo Leaders: please don’t hog them for yourselves!

If you have any questions or concerns, please directly contact Pearson/Prentice Hall’s Heather Fox at heather.fox@pearson.com Also, for those teams who are not approved or yet to be approved, you can still score a rather nice 35% discount on the books by registering your LoCo with the Pearson User Group Program.