Jorge's Stompbox

Plug in, crank it to Eleven.

Juju Ecosystem Status for 22 May

Here’s all the goodies for the week:

Updates this past week:

vUDS

  • Charm Auditing! Marco will be providing list of things and charms not up to snuff, post to list in order to get fixes (or eventually remove from store)

Blueprints Discussed:

Please check out the blueprints, there’s a ton of detail there!

Charm Tools

  • Fixed dependency to recommend juju-core or juju, fixes Jono’s bug.
  • charm-helpers being split into it’s own project: https://launchpad.net/charm-helpers
    • Rewriting a bunch of them into python instead of a mishmash of bash and python, gives us cross-OS compatability, better templating, easier testing.
    • Rolling out a single charm-helpers package

Docs

  • Pre-beta live site (Nick hates it when we link it. :))

Charm Testing

  • Rewriting the jitsu test code so it works as a juju plugin to enable easier testing.
  • lp:juju-plugins

Charm Framework Updates

  • node.js - Jeff Pihach linked up with Mims, experienced node.js dev. Good things on the roadmap here. We’ll get a better status when Mims returns from Gluecon
  • rails/rack - Follow up with Pavel?
  • Django, someone mentioned in UDS that it’s nearly ready to be submitted to the store.

CFPs & Upcoming Events

  • Mark Mims is at Gluecon! Go get em!
  • Submitted to Strata in NYC. (mims)
  • Strata in London submission in progress (jamespage)
  • TexasLinuxFest, arosales to present.

Charm Schools

  • Next Friday is Part 2 of “How to write a charm”
  • We’d like to have a roadmap for charm schools
  • Over the next day or so Jorge to publish a schedule for charm schools, will be on the Events.
  • We’d like to be responsive to user needs.
    • Keep biweekly cadence, be flexible enough to do on-the-spot charm schools.
  • Jorge to add more detail to charm schools on the web page, show what topics were covered in more detail.

  • capture the IRC Logs (duh!).

  • Topics people want: Puppet/Juju, Charming from Scratch, Improving an existing charm (including the workflow to submit it back)

Open Source Is Not a Sport for the Armchair Quarterback

Earlier this week the techboard asked what we should do with Brainstorm. Having been involved with Brainstorm since almost the beginning, I felt it appropriate to handle how we would deal with it since no one wants to be unpopular, except for me of course. The TLDR is that Dell launched IdeaStorm and of course people thought this would be a great idea for OSS.

The very first thing I noticed when the idea of shutting it down was a fundamental misunderstanding of what Brainstorm is and is not. So let me be clear here:

Brainstorm was never about user-driven voting for what goes into Ubuntu.

Brainstorm was about communicating ideas that the user base were interested to Ubuntu, and at THAT it did a pretty decent job. Every cycle the tech board was taking in the top ideas and responding to them. Most of these ideas were pretty obvious. Ubuntu developers don’t need anyone to remind me that Ubuntu needs to do a better job at hardware support. We know and deal with these issues every day.

Brainstorm was about engaging developers with users, and here’s why that doesn’t work anymore:

  • Just go to UDS. It’s virtual, anyone can join without caring about travel expenses, just talk to developers directly.
  • Be involved in projects you care about; there’s mailing lists and tons of feedback options for developers.
  • It takes a reasonably intelligent person about 10 seconds to come up with 10,000 years of development work that will never be accomplished with the resources we have.
  • Go do stuff, the more you do, the more you get a say.
  • At the end of the day I’m swimming in great ideas. I don’t need great ideas, I need people willing to make great ideas a reality.

It seems that a great number of people think that Brainstorm is all about “wish-driven development” - the idea that you will come up with an awesome idea and then a team of developers will go do that for you and deliver what you want. Unfortunately that is not how it works. The only way you will ever get things done is if you do the work alongside other people. The currency of Open Source is the amount of work you’re willing to put into it. And while some people are saying that they’ll move to other distros or give up on Ubuntu because “no one listens to me” are in for a rude awakening when they realize that no open source project is driven by webpoll results.

Some people have equated the “age of Unity” as the reason as to why Brainstorm is failing, but I don’t think so, the site was flailing long before then. I might be seemingly overly negative, and that’s not my intent. In fact the barrier to get involved with Ubuntu is lower than ever.

The ironic bit so far is that the amount of complaints about Brainstorm shutting down far outnumber the amount of volunteers who have laid aside a ton of their own personal time to do the work to make the site succeed. That tells me a few things. First of all, the amount of people who will complain that things don’t work like they want them to is high. The amount of people willing to work on Ubuntu to fix these problems is relatively low.

So is Brainstorm a failure? Probably. I think we learned a bunch of things. No other OS has tried this before. Sure, they say Windows 7 was my idea, but you know that’s made up. I like that we tried, shrug.

I like that we now do design and user-feedback based improvements into Unity. Some people don’t like that. Some people don’t like that we do test driven development either. To each their own. Anyway Brainstorm was never my idea, it was a community idea that seemed to make sense at the time and whose course has run. Let’s torpedo the unrealistic idea that webpolls run an operating system and just people wired into making an operating system. Want to make a difference? Here’s the schedule for the last day of UDS if you want to get involved.

How I Will Remember Jeff Hanneman…

If you don’t know .. the one on the left …. I’ve backed it up a bit because this is one of the best metal riffs of all time, and it’s better to appreciate both Jeff and Kerry together.

Here’s the video.

I Wish Nest Did More Than Thermostats

There is nothing more awesome in life than to watch technology fail people. Here’s the UI to my sprinkler system to my lawn:

Here’s what a 45 node deployment of OpenStack on Ubuntu looks like:

So at some point running a Linux cloud became easier than managing water coming out of tubes in the ground.

For some reason my sprinkler is running at 6pm instead of 6am. I don’t know how to fix it because:

  1. The UI is so terrible I want to punch someone in the neck.
  2. The clock is wrong. I don’t know how to set the clock manually without having to go through point #1. Why I’m setting a clock manually in 2013? No clue.

If Nest made a lawn sprinkler system it would show a brown patch in the UI. If I turned the knob to the right it would turn greener and greener until it matched my cheapness-to-green ratio. It would check the weather so it wouldn’t water my lawn when it’s raining like this thing does. It’s raining, why are you on? Because some sprinkler company bolted on the world’s worst UI to what amounts to a … cron job.

For bonus points it would also check Google Maps and compare how green my lawn is to my neighbors, so I can keep up with the Joneses without spending too much. It would know when there’s a water shortage and talk to my neighbor’s sprinklers and the water company to Do The Right Thing for the neighborhood.

I love that my Nest is so smart that people think it’s just a dumb knob. What it’s doing in the background is the real brains, and I love that I don’t have to think about it. I hope they move on from thermostats and bring this kind of stuff to other household appliances. You should see the buttons on my dishwasher.

Juju Charm Store Updates for 1 May

Meeting Details

  • Hangout URL: https://plus.google.com/hangouts/_/c90e71c6c9f5031e7b771639e8c0ea793a645663?authuser=0&hl=en
  • Video recording: http://youtu.be/6nB3isrvAaw
  • Pad: http://pad.ubuntu.com/7mf2jvKXNa
  • Status Board: https://trello.com/board/charmers-board/4ec1696da3f94bd2ea5b2b01

## Updates this past weeK:

Charm Tools

  • Marco to post status/instructions of charm-tools to mailing list.
  • instructions on how to add categories and icons.

Docs

  • Aiming to have charm section to be done tomorrow (2013-05-02)!
  • Beta docs coming shortly, Getting Started is done.
  • Developer documentation to come after user docs.
  • hoping to get lots of feedback from Beta

Juju Images to reference on HP Cloud

  • Ben Howard/smoser working this so we don’t have to keep updating the config files for new images. (simple stream data generated in the future).

    vUDS 1305

  • Email any topic ideas to ubuntu-server list

Charm Testing (Marco)

  • Drastically improved, but not 100%. There’s still some false positives, hope to finish that this week.
    • About 85-95% accurate currently.
  • Jenkins Status

Charm Framework Updates

  • Mims @ Rails Conf this week in Portland
    • Mims texted Jorge that he’s had decent contact with people @ Railsconf wrt. the rails charm.
  • Always looking for Java stacks.

CFPs & Upcoming Events

  • LISA13 - DONE!
  • Strata Europe - Jamespage to submit
  • Strata NY - Jamespage to submit
  • Ben Howard @ OpenWest Conference in Utah (May 3rd) - jorge to events list.
  • Juan @ Cisco’s Open Conference
  • Join Mark Mims @ Gluecon to talk Juju!

Misc

  • Matt Griffin from Percona dropped in. James Page to link up with him at vUDS. (Jorge too).

13 Reasons to Deploy With Ubuntu Server (Part 3)

LXC/Containers and Vagrant

It seems that a bunch of the things I’ve been talking about so far are ops focused. Juju straddles both dev and ops, it’s why we call it “DevOps Distilled”. I’d like to point out some of the nice things that are useful for developers as well.

The first is our LXC support. You can think of Linux containers as “super chroots”, that let you segment whatever you want on a machine without the overhead of virtualization. We can use it to segment your deployed services with Juju, or you can use them to play with things in a nice little sandbox. And yes, we’ve got our eyes on Docker too.

Speaking of sandboxes, we also now provide official Vagrant images so you can quickly fire up server instances on any OS to do your development on. And just like the rest of our cloud images, frequently updated and fully supported.

Landscape

Organizations need to manage large fleets of Ubuntu systems, and that’s where Landscape comes in. As the usage of Ubuntu server to deploy services scales out seamlessly through Juju, your infrastructure needs day-to-day tending to.

Landscape lets you manage thousands of Ubuntu servers with the same ease you would manage one - you can fix a security issue affecting hundreds of machines with a single click, and what’s more you can prove you did to your compliance or governance team without having to spend time creating the paper trail yourself - that is a really big deal if you work in an IT department and don’t enjoy spending your time creating reports.

Landscape is designed from the ground-up to manage Ubuntu systems, and that reflects on the tight integration between Ubuntu core components and Landscape. In Ubuntu, we use apt to update systems from the shell - Landscape talks to apt, not a different custom backend, so if you know how apt behaves, you already know how Landscape does it - this is a design style that makes the learning curve for Landscape much nicer than other tools’ - and makes it inherently shell-compatible, for we understand very well that an administrator’s debug tool of choice is SSH.

Landscape’s management goodness is exposed through its server’s API - providing you with a robust toolset of management actions that are maintained for you across all current Ubuntu releases, from cutting edge Raring to reliable and dependable Precise - any architectural differences are taken care of for you, so you don’t have to.

Landscape is built for enterprises customers, and it has both SAAS and Dedicated Server editions. A free 30-day trial is available for those who want to try first hand.

Ubuntu and its community

Sounds kind of cheesy, but Ubuntu itself is a feature.

  • There are no “enterprise” and “free” versions of Ubuntu. It’s all just Ubuntu, and it’s all free to the end user. That’s right. We don’t artificially split the OS. If Ubuntu works out of the box for you and you’re skilled enough to use it, you’re good to go. You get the exact same OS, with the exactly same support that paying customers get. This does motivate us to provide excellent support options (see below).

Run into a problem? Well you can call us and get support, all without having to reinstall from the “free version” to the “paid version”. No shuffling licenses either or caring about which of your systems are supported or covered, and no waiting for security fixes to trickle down to your free version. Just pay for what you use.

  • cloud-init - You can control instance initialization with cloud-init. It’s like preseeding/kickstarting a server, but in the cloud.

  • The community - the massive resources of the Ubuntu community is available to you. And it’s not just about support, it’s about things like a wide range of PPA archives of contributed packages and ecosystem of packages for things that run on Ubuntu, but might not be part of the distro itself.

  • Built on Debian. `Nuff said. So you get nice things like tasksel tasks, allowing you to simply install Mail, Web Server, or LAMP stacks with one checkbox and the breadth of software of the Debian archive.

  • A predictable and solid release cycle, with 5 years of support for our LTS releases.

Ubuntu Advantage

As I mentioned above, since Ubuntu is free-to-use we are extremely motivated to provide service that offers a good value. Our service offerings are split into two areas, server and cloud:

Here are our prices for traditional servers, starting as low as $320 per box. And here are our cloud prices. You can also add on Landscape and what we call Premium Service Engineers. PSEs are our best-of-the-best. If you need an expert to solve your toughest problems, you can add on a PSE.

Shop around and check out our competitor’s too, you’ll find that our prices very competitive.

You’re in fine company

Along with Netflix, Wikipedia, Inktank, AT&T, HP, Dreamhost, Rackspace, Instagram, Dropbox, SmugMug, Samsung, NTT, Deutsche Telekom, 10gen, and Amazon. They, and many others, chose Ubuntu Server because of some or all of the things I mentioned above.

So that’s the jist folks. As you can tell we’re pretty excited about the things we’re bringing to the cloud and your server room. We’re one year away from our next Long Term Support (LTS) release and committed to bring all of these technologies to bear. Some of them have been in the cooker for a bit (MAAS and Juju) and are still under heavy feature development, so if you were an early adopter and been burned by a bug or lack of a feature, now is the time to start looking at it again and giving us feedback. As the Fall approaches we’ll be shifting to getting ready for 14.04 LTS and keep on pushing the envelope of the cloud.

Check out the previous sections if you missed them:

13 Reasons to Deploy With Ubuntu Server (Part 2)

I started off yesterday detailing the first four reasons why you’d want to consider deploying with Ubuntu Server. Here are today’s four.

Juju and its charms

And this is where it starts to comes together. You’ve got your OpenStack running with tons of nodes, exabytes of storage on Ceph, now what? No one builds a cloud for the sake of building one. You have work to do, and that’s what Juju is all about.

Juju takes all this infrastructure you just set up and makes you… not care about it. Juju bucks the trend of thinking about machines and instances as a whole. We back you up and show you a higher level view of your deployment and get you to think about what’s really important to your business, deploying services.

So now you think of your deployment as environments, like Hadoop, Cassandra, MongoDB, or $your_application. You tell Juju how to model your deployment, and it uses all the things I just mentioned, MAAS, OpenStack, your public cloud provider, to make your environment. You manage things at the service level. Just assemble your blocks and the juju scripts (we call them charms) do the rest.

So what’s that mean for you? Here’s an example of how you can deploy your node.js app or your rails app. What you see there is our end goal for everyone who wants to deploy anything to the cloud - you should be able to do so with a few commands or a couple of mouse clicks. We’re well on our way with over 120 services available for you put together, and plenty more on the way. All of them are Free Software, ready to be improved upon and shared with the wider community.

Here’s an example of setting up sharding with MongoDB if you really want to see how Juju takes the complexity out of service orchestration.

Hardware Certification

We’re certified to run on hardware from the top names in servers. Dell, HP, Lenovo, IBM, and Acer are just some of our certification partners. There’s not really much else to say. This bullet seems really boring, but it’s certainly not boring for the people in our hardware labs, this is just something most sysadmins will expect to work out of the box.

ARM

We’ve been building Ubuntu on ARM as a fully functional, well tested OS that can support multiple SoCs (Calxeda, TI, Marvell Armada XP, and anything Ubuntu Touch/Client uses) – and have been doing so for over 4 years.

We’re working with Calxeda on their ECX-1000 EnergyCore Products (Highbank). We are going to be using the ECX-1000 as our ARM server reference platform for 13.04.

And we’ll be working with them on their “Midway” product refresh when that hits.

Ubuntu Guest on Public Clouds

Whether it’s like Amazon Web Services, HP Cloud, Rackspace, Internap, or Microsoft Windows Azure, you’ll find official Ubuntu images, and you’ll find that the per-hour cost to you is $0.00.

The nice thing we do in the public clouds is that we work closely with the cloud providers to give you a great experience. Our images are directly published into these clouds as part of the release process, they’re not made after-the-fact. And since it’s Ubuntu, it’s the same on every cloud, you won’t find different behavior from your local servers or from vendor to vendor, it’s important to us to provide you a consistent experience.

On the certified public clouds you’ll not only find regularly refreshed and supported images, you’ll find other goodies, like built-in local mirrors so that your updates are LAN speed in your cloud deployments. And since we partner with these providers and continually work with them to improve on this experience you’ll always be getting the best available service.

Check out the previous sections if you missed them:

13 Reasons to Deploy With Ubuntu Server

(Part 1 of 3)

Sometimes people ask me why they should use Ubuntu Server. It’s an understandable question, after all, Ubuntu gets a bunch of attention on the desktop (and more recently mobile), but people tend to forget that Ubuntu is an excellent server distribution, quietly humming along helping to run some of the world’s coolest companies at scale.

So I thought I’d document why you should choose Ubuntu for your organization. Paul Hammond from TypeKit has said “No startup has ever failed because they picked the wrong Linux distribution.” TypeKit’s Linux of choice? Ubuntu. Why is that? So I started to think of reasons why you would choose Ubuntu, and to celebrate the release of 13.04, I came up with 13 major reasons you would choose to go with us. Let’s get started, in no particular order! I’ll split this post over the next few days so you can digest it easier, and more importantly, go play with the things I talk about!

OpenStack and the cloud archive

As I said a few weeks ago, shipping a well tested and robust OpenStack is one of our primary missions. We’ve been shipping OpenStack in Ubuntu for over 3 years now, and every cycle we get better at it. We do daily and per commit automated testing of OpenStack. Maybe that’s one of the reasons Ubuntu is the reference OS for OpenStack.

We’re committed to backporting the latest OpenStack releases to Ubuntu 12.04 LTS (Long Term Support). That means if you want the fresh upstream goodness of OpenStack but want a stable platform, you can get that with Ubuntu; no one else is doing anything like this.

How ready is OpenStack and Ubuntu for production? It’s there, and we know that because we’re running OpenStack in production. Moving from a traditional IT setup to “devops” using the cloud is tough, and we’re still in the process of doing it, we learned a bunch of best practices that we’ve shared with the community, see Robbie Williamson’s talk: OpenStack in Production: the Good, the Bad & the Ugly

Ceph

Ceph is a massively scaleable distributed file system that runs on commodity hardware. And it comes with Ubuntu in main and is fully supported by Canonical and Inktank. Here is the blueprint of the usecases we are enabling out of the box. We’re also working on getting per-commit and daily automated testing of Ceph as well.

I am doing Ceph a disservice by only mentioning it in one paragraph. The sheer capabilities of Ceph and it’s scalability can take up pages, so if you’re looking to get started with Ceph, check out this tutorial from Inktank on how they deploy Ceph with Juju and check it out for yourself.

MAAS

On the local server front, we provide a provisioning tool called MAAS, which stands for Metal-as-a-Service. MAAS not only provisions bare metal machines for you (on demand), but we like to say it “brings the language of the cloud to physical services”. Sometimes you just want the ability to horizontally scale as demand comes in, and have machines turn on, install the OS, and then get to work, all automatically.

We use MAAS to deploy OpenStack, or you can use MAAS to deploy whatever servers you need, e.g. Hadoop or Ceph, on whatever scale-out hardware you have, e.g. Intel, AMD, or ARM, all programatically. It’s a simple but powerful tool to deliver provisioned servers to you.

Hardware Enablement Stack

Ubuntu provides what we call an LTS Enablement Stack.

Simply stated, we provide newer kernels for the Long Term Support release, that means if you end up with newer hardware that needs newer hardware support you can still use an LTS release and have a stable userspace. The enablement kernels then provide you an upgrade path for the next LTS.

For more tips on how to use these stacks, check out this blog post.

Check out the previous sections if you missed them:

Juju Charm Updates for 13.04

Well it’s another cycle and another Ubuntu release and it’s another cycle’s worth of Charm Store improvements. Since the Charm Store doesn’t freeze we can continually improve the capabilites of the charms over time while users can stick to the stable userspace that 12.04 LTS provides.

If you’re new here and have no idea what Juju is then check out the video. And what are charms? Well charms are the services you deploy. If juju is apt-get for the cloud, then charms are the packages for those clouds.

Juju has one mission in life, and that’s getting your services not only deployed, but manage them over the course of their lifecycle. Our job is to take your code and get you up and running on AWS, HP Cloud, Azure, and OpenStack as easily as possible.

First the raw numbers

  • 133 peer-reviewed charms in the store
  • 288 charms in personal branches
  • 151 total contributors (82 Canonical, 57 non-Canonical, 12 Canonical Juju coredevs)

As far as raw Charm download numbers, these are currently unavailable to browse, however next week or so when the new GUI/website land users will not only be able to see the number of downloads per charm, but if the charm works on a given provider. More about that in a future blog post …

So what’s in the app store?

Rails, node.js, and Django

One term you might be hearing tossed around is “PaaS on my own terms”. The idea here is that people really like the workflow that you can get with Heroku and other PaaSes, but don’t want the added complexity. Just your app and your provider.

Here’s how it works… assuming your Rails app is “myapp”, let’s deploy your application with postgresql to show you the kind of workflow we provide. We provide generic “platform charms” that let you snag things right out of version control and deploy them. Assuming you have a myapp.yaml file describing your application:

myapp:
  app_name: myapp
  repo_type: git
  repo_url: "https://github.com/mmm/testrails.git"

Now you’re to deploy on your cloud provider, this can be AWS, HP Cloud, your own OpenStack, or your local Ubuntu laptop:

juju bootstrap  
juju deploy --config ~/myapp.yaml rails myapp
juju deploy postgresql

relate them

juju add-relation postgresql myapp

open it up to the outside world

juju expose myapp

Find the myapp instance’s public URL from

juju status

Like all services that can connect to a reverse proxy, we’ve made it so that you can easily scale out your application horizontally via haproxy:

juju deploy haproxy
juju add-relation haproxy myapp
juju add-unit -n5 myapp
juju expose haproxy  # Change your DNS to point to this IP instead
juju unexpose myapp  # We’re behind haproxy now, no need to leave this open to the world.

And this is just the rails example, we can do this for node.js and django, and I’m always keeping an eye out for Java-smart developers, so if you want to make this kind of workflow for your platform, let me know!

Databases and other infrastructure

Need master/slave with that Postgres? We’ve got it, you just need to fire it up with a few commands.

Any Ubuntu user can set up a master/slave Postgres setup this way. It’s nice, it’s like having Stuart Bishop and his roving band of IS Postgresql gurus setting up your Postgres config for you. And since we dogfood OpenStack and our charms everyday, it’ll work.

Need to set up replicasets in MongoDB? We’ve got that built into the MongoDB charm too.

Or sometimes you just need a blank LAMP stack or you can toss in some elasticsearch if you want it.

Come see us and tell us what you need

Come check us out at any one of these events in 2013 if you’re interested in learning more or join us at our next virtual Charm School.

Weekly Juju Charm Updates for the Week of 24 April

Agenda Items: - Questions - Charm Testing - Charm Docs - Framework Charms - Workflow

Questions from the Community

  • Rick and Aaron - How do relations between charms work so we can show it on the website/GUI?
    • We want to show which charms relate to what so users can discover which charms go “good” with other charms.
    • If you are on the apache page you’d show Wordpress, not nginx, etc.
    • Do we want to show most downloaded, most reviewed?
    • We need a definition of what a “related charm” is.
    • Mims - 2 sides need to implement the same interface,
    • Sort by quality or downloads? Provides vs. Requires?

We’ll walk the interfaces graph, show all charms, sort by reviewed/download/ratings as related.

  • Nelson Roberts joined the hangout and asked about Juju/Rackspace; openstack provider works, waiting on their cloud to be more vanilla, we should Just Work on there soon.

Meta Data Categories and Icons in Charms.

  • Charm Proof gives a warning
  • Policy does include meta-data for categories, we need to also update for icons
  • https://juju.ubuntu.com/docs/policy.html

Charm Testing

  • reliable testing for graph and embedded! Go Marco!
  • https://jenkins.qa.ubuntu.com/view/Precise/view/Precise%20Charms/
  • embedded test will run, if one is not available then a basic smoke test (charm goes to running) is ran.
  • To go live with the GUI/jujucharms.com soonish, all integrated so it’s more in our face than off in jenkins somewhere.

Docs

  • Docs branch:- https://code.launchpad.net/~evilnick/juju/go-juju-docs
  • Will go live soonish, not for 13.04.

Kapil’s new API tool:

http://launchpad.net/python-jujuclient

Framework Charms

  • Not many updates, team is concentrating on 13.04 readiness/switchover/relaunching stuff.

Get ready for UDS:

  • http://summit.ubuntu.com/uds-1305/
  • http://uds.ubuntu.com/register/
  • https://wiki.ubuntu.com/UDS/Create