tag:blogger.com,1999:blog-21729491992029040072024-03-27T19:23:25.787-07:00A Grimm's RealityAndy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-2172949199202904007.post-5267637188390056412013-04-19T07:42:00.000-07:002013-04-19T07:42:07.396-07:00Cloud-in-a-box-in-a-VM (in a nutshell)For the past few months, one of my projects at Eucalyptus has been our CentOS 6 based "Silvereye" (a.k.a. FastStart) installer, which assembles multiple repositories, custom install classes, and a few helper scripts onto a DVD-sized ISO image. One of the challenges has been that I'm a remote employee, and downloading an ISO file from our Jenkins server takes too long. At the same time, loading the ISO into Cobbler doesn't really test the same code paths as booting a DVD. So how can I test my cloud installer? Nested virtualization, of course!<br />
<br />
As of Fedora 18, I've found nested KVM to be fairly reliable. The performance isn't earth-shattering, but it's usable. There are several things about the setup that may not be obvious, though, so I'll walk through everything I did for my test machine.<br />
<br />
The server I'm using has two Xeon CPUs, 8 GB of RAM, and 2 TB of disk. I've allocated 50GB of LVM space for the root filesystem, and when I create new VMs, I give them a 100GB logical volume as their disk ( lvcreate -n vmName -L 100G vg01 ).<br />
<br />
The server is on a 10.101.0.0/16 network, and I "own" 10.101.7.0/24 within that, as well as 192.168.57.0/23 for "private" addressing. I've connected bridge device br0 to my primary ethernet device, em1:<br />
<ul>
<li>/etc/sysconfig/network-scripts/ifcfg-br0
<pre>DEVICE=br0
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=dhcp # IP is reserved: 10.101.1.25
TYPE=Bridge
DELAY=0
PERSISTENT_DHCLIENT=yes
</pre>
</li>
<li>/etc/sysconfig/network-scripts/ifcfg-em1
<pre>DEVICE=em1
ONBOOT=yes
NM_CONTROLLED=no
TYPE=Ethernet
BRIDGE=br0
</pre>
</li>
</ul>
I enabled on the system by placing "options kvm_intel nested=1" in /etc/modprobe.d/kvm.conf and reloading the kvm_intel module.<br />
<br />
To allow the VMs to be connected to the bridge interface, I've added "allow br0" to /etc/qemu/bridge.conf<br />
<br />
I also set /proc/sys/net/ipv4/ip_forward to 1, so that packet forwarding works.<br />
<br />
For now, I've completely turned off iptables. This is not ideal, but I'll leave iptables rule building for another post. <br />
<br />Booting an image into the FastStart installer looks like this:<br />
<br />
<pre>qemu-kvm -m 2048 -cpu qemu64,+vmx -drive file=/dev/vg01/ciab,if=virtio \
-net nic,model=virtio,macaddr=52:54:00:12:34:60 -net bridge,br=br0 \
-cdrom silvereye-nightly-3.3-m5.iso -boot d -vnc :1
</pre>
<br />
Let's look at the individual pieces of that.<br /><br />1) "-m 2048" is 2GB of RAM<br />2) "-cpu qemu64,+vmx" specifies that we are emulating a CPU capable of virtualization<br />3) "-drive file=/dev/vg01/ciab,if=virtio" is our LVM-backed "disk"<br />4) "-net nic,model=virtio,macaddr=52:54:00:12:34:60" is our virtual network interface. Specifying a unique MAC address is <b>very</b> important! If you don't do this, every VM will get the same MAC, and they won't be able to communicate with each other. They <i>will</i> all be able to send and receive other traffic, though, which can be maddening if you don't realize what's going on.<br />5) "-net bridge,br=br0" says to connect the host TAP device to bridge br0. This gets you non-NAT access to the physical network, since the bridge is also connected to em1.<br />6) "-cdrom silvereye-nightly-3.3-m5.iso -boot d" connects our ISO image and boots from it.<br />7) "-vnc :1" starts a VNC server on the host (on port 5901) for this VM's console.<br />
<br />
I connect to 10.101.1.25:5901 with a vnc client, and proceed with the install, selecting "Cloud in a Box" at the boot menu. Since I own 10.101.7.x, I give this VM the IP 10.101.7.1, and configure its "public IP range" to be 10.101.7.10-10.101.7.20. The subset of IPs here is arbitrary; I wanted to leave some for my other test clouds. For private IPs, I'll use 192.168.57.0/24, half of my allotted range. The default gateway and nameserver are exactly the same as for the host system. The host bridge is just a pass-through, not an extra routing "hop".<br />
<br />
<br />
<br />
<br />
Once installed, I can access the cloud through the normal channels -- SSH, the admin UI on port 8443, and the user console on port 8888. I test by logging into the user console, generating and downloading a new key, launching an instance, and SSHing into the instance's public IP.<br />
<br />The chain of devices through which data flows may not be entirely clear. When you launch an instance inside the virtual cloud on 10.101.7.1 -- and let's say for the example that it gets an IP of 10.101.7.10 -- here's the list of "devices" through which packets flow when you connect to it from a different physical system:<br />
<ol>
<li>the physical interface of the host, em1</li>
<li>the host bridge, br0</li>
<li>the host tap interface</li>
<li>the VM's "eth0" interface</li>
<li>the VM's "br0" bridge (which is <i>not</i> enslaving eth0 in this case)</li>
<li>the VM's tap interface (vnet0)</li>
<li>the nested VM's eth0</li>
</ol>
The only place that NAT happens here is between 4 and 5, and that's only necessary because I chose to use eucalyptus's "MANAGED-NOVLAN" mode.<br />
<br />
So that's a cloud-in-a-box-in-a-VM (in a nutshell).<br />
<br />
For a more traditional deployment, simply boot more VMs into the installer using the same qemu-kvm command format mentioned above, choosing "Frontend" for one, and "Node Controller" for the other(s). For each one you boot, you need to create a new lvm volume and change the MAC address and the vnc port to avoid conflicts. When running multiple clouds, you should also make sure that your IP ranges never overlap (i.e., don't let two clouds use the same public IPs or private subnet range).Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com4tag:blogger.com,1999:blog-2172949199202904007.post-24691040905125470812013-01-03T14:18:00.001-08:002013-01-03T15:48:36.562-08:00Using aws-cli with EucalyptusJust before the holidays, Amazon released <a href="http://aws.amazon.com/cli/" target="_blank">awscli</a>, a new command-line interface for managing AWS resources. The code is based on <a href="https://github.com/boto/botocore" target="_blank">botocore</a>, the core python library for the next major version of <a href="https://github.com/boto/boto" target="_blank">boto</a>. I took awscli for a spin to see if it worked with the <a href="http://www.eucalyptus.com/eucalyptus-cloud/community-cloud" target="_blank">Eucalyptus Community Cloud</a>, and as is often the case, the answer was ... almost.<br />
<br />
First, it's useful to understand the fundamental problems that awscli was trying to address. The most obvious is profiles. Cloud users deal with multiple regions, accounts, users, etc., and keeping separate configurations for each one is a hassle. awscli uses a section-based config file format which allows for multiple profiles, each of which can reference it's own region, access keys, etc.<br />
<br />
Another problem that this new code solves is the centralization of region and service data into JSON files which are easy to read, write, and parse. See <a href="https://github.com/boto/botocore/blob/develop/botocore/data/aws/_regions.json" target="_blank">_regions.json</a> and <a href="https://github.com/boto/botocore/blob/develop/botocore/data/aws/_services.json" target="_blank">_services.json</a> in botocore for examples.<br />
<br />
What I found was that rather than trying to alter the existing data files, what I really wanted was a eucalyptus "provider" with its own JSON files. I'll spare you all my trial-and-error, and simply explain what worked:<br />
<br />
<ol>
<li>git clone https://github.com/boto/botocore.git</li>
<li>git clone https://github.com/a13m/aws-cli.git (note that this is my fork -- upstream is https://github.com/aws/aws-cli.git )</li>
<li>Install botocore and aws-cli however you prefer ( I use "python setup.py install --user" in each directory)</li>
<li>create a provider data directory, and a "euca" directory inside it. I'll use /var/tmp/providers as the top directory.</li>
<li>create <a href="https://github.com/a13m/aws-cli-provider-data/blob/master/ecc/_regions.json" target="_blank">_regions.json</a> and <a href="https://github.com/a13m/aws-cli-provider-data/blob/master/ecc/_services.json" target="_blank">_services.json</a> under the "euca" directory (the linked examples here should work for ECC verbatim)</li>
<li>symlink to botocore/data/aws/ec2.json and botocore/data/aws/iam.json in the euca provider directory </li>
<li>Create your ~/.awsconfig file (or whatever you'd like to call it):<br />
<pre> </pre>
<pre>[default]
aws_access_key_id=XXXXXXXXXXXXXXXXXXXX
aws_secret_access_key=XXXXXXXXXXXXXXXXXX
region=ecc
provider_name=euca</pre>
<pre> </pre>
</li>
<li>export AWS_CONFIG_FILE=$HOME/.awsconfig</li>
<li>export AWS_DATA_PATH=/var/tmp/providers</li>
<li>try some commands, such as:
<pre> </pre>
<pre>aws ec2 create-volume --size 1 --availability-zone partner01
aws ec2 describe-volumes
aws ec2 describe-images</pre>
<pre> </pre>
</li>
</ol>
It may take a couple of iterations for the patch I've proposed to be accepted upstream, but in the meantime, I hope this is useful information. As I've mentioned in the <a href="https://github.com/aws/aws-cli/pull/23" target="_blank">pull request</a>, the solution is not ideal, as it requires that your default profile in a config file reference the euca provider, but I went for the least invasive fix first. Note that even with this version, you can use profiles to group all of your eucalyptus cloud credentials into a single config file, and then have a second file for AWS profiles. Switching back and forth is just a matter of setting AWS_CONFIG_FILE.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com7tag:blogger.com,1999:blog-2172949199202904007.post-78639423923633792662012-08-15T09:22:00.001-07:002012-08-15T09:22:36.682-07:00Fun with GitHub pull requestsGitHub pull requests are great for contributors. They offer a very simple way to publicly post a patch against a piece of open source software and get feedback from the maintainers. For various reasons, though, actually doing the merge of a pull request via GitHub's UI may not be the ideal thing for a project maintainer. Reasons include:<br />
<br />
* a policy of putting the commit through test prior to merging it<br />
<br />
* amendment of a relevant issue ID to a commit message which lacks one (I've not done this to a contribution, but I've considered it)<br />
<br />
* the pull request should be merged into a branch other than the one it was filed against<br />
<br />
and I'm sure you can think of others. So the first couple of pull requests that I accepted were a bit painful, because my process was:<br />
<br />
* git clone the contributor's repo<br />
<br />
* use git format-patch to export the patches<br />
<br />
* use git am to import the patches<br />
<br />
In particular, the git clone of an entire repo just to get a 5-line patch seemed like a tremendous waste of time and bandwidth, and I knew there had to be a better way. Obviously github's website displays the diff associated with the pull request, so it has to be stored _somewhere_, right?<br />
<br />
And of course it is. For example, here is a pull request:<br />
<br />
<a href="https://github.com/eucalyptus/eucalyptus/pull/3/">https://github.com/eucalyptus/eucalyptus/pull/3/</a><br />
<br />
To see the properly formatted patch associated with it, simply drop the trailing slash and add ".patch":<br />
<br />
<a href="http://www.blogger.com/%20https://github.com/eucalyptus/eucalyptus/pull/3.patch" target="_blank">https://github.com/eucalyptus/eucalyptus/pull/3.patch</a><br />
<br />
It's worth noting that I found this via a link tag which exists in the source of the pull request page:<br />
<br />
<link rel='alternate' type='text/x-patch' href='/eucalyptus/eucalyptus/pull/3.patch' /><br />
<br />
So there's probably some code floating around somewhere to simply follow that link instead of knowing how to alter the url. Anyway, I'm quite happy to have found this. I'm not sure why there's not an obvious link to it in each pull request page. I think it would serve project maintainers well.<br />
<br />Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com4tag:blogger.com,1999:blog-2172949199202904007.post-79919264380901397842012-07-03T13:57:00.000-07:002012-07-03T13:57:14.635-07:00Jira full text search tipsOne of the advantages of using <a href="https://eucalyptus.atlassian.net/" target="_blank">Jira at Eucalyptus</a> is that it has very good
<a href="http://lucene.apache.org/core/" target="_blank">Lucene</a>-based full-text search. It's not necessarily obvious how to use
it, though. If you want to search for a multi-word string, you have to
use the advanced search (<a href="https://confluence.atlassian.com/display/JIRA/Advanced+Searching" target="_blank">JQL</a>), and quote it like this:<br />
<br />
text ~ "\"disk space\""<br />
<br />
if you only want to find the individual words rather than the exact string, remove the escaped quotes:<br />
<br />
text ~ "disk space"<br />
<br />
The first query returns about 4 results for me, while the second returns dozens, so the subtle difference can be very important.<br />
<br /><a href="https://confluence.atlassian.com/display/JIRA050/Performing+Text+Searches" target="_blank">More search tips here.</a><br /><br />
Just be aware that most of those tips don't seem to work via "quick
search". You have use them inside quotes in a JQL search. I tested
things like:<br />
<br />
text ~ "\"disk space\"~10" (disk and space within ten words of each other)<br />
text ~ "behavio*r" (behavior/behaviour plus other possible but unlikely strings)<br />
<br />
and so on. You can also search specific text fields such as:<br />
<br />
summary ~ "behavio*r"<br />
<br />Hopefully this info will make it easier for people to find existing issues in our Jira instance as well as others around the web.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com4tag:blogger.com,1999:blog-2172949199202904007.post-27087752517575906762012-05-03T12:49:00.001-07:002012-05-03T12:51:10.790-07:00Sampling GitHub API v3 in PythonEucalyptus is in the process of moving code to <a href="http://www.github.com/" target="_blank">GitHub</a>, and this week I finally decided to look at the available API tools for working with GitHub. I wanted a tool written in python, since that would be the fastest for me to extend, and I found <a href="http://packages.python.org/github2/" target="_blank">github2</a>. Unfortunately, that homepage had a prominent warning that the code only worked with GitHub's old APIs, which were being turned off this week. So I decided to investigate what I could do from scratch in a small amount of code. I had already started using <a href="http://pypi.python.org/pypi/restkit" target="_blank">restkit</a> in <a href="https://bitbucket.org/agrimm/jiranemo/overview" target="_blank">jiranemo</a>, so that seemed to be a reasonable starting point. Here's what I came up with:<br />
<br />
<br />
<pre class="brush: python">import json
from restkit import Resource, BasicAuth, Connection, request
from socketpool import ConnectionPool
pool = ConnectionPool(factory=Connection)
serverurl="https://api.github.com"
# Add your username and password here, or prompt for them
auth=BasicAuth(user, password)
# Use your basic auth to request a token
# This is just an example from http://developer.github.com/v3/
authreqdata = { "scopes": [ "public_repo" ], "
note": "admin script" }
resource = Resource('https://api.github.com/authorizations',
pool=pool, filters=[auth])
response = resource.post(headers={ "Content-Type": "application/json" },
payload=json.dumps(authreqdata))
token = json.loads(response.body_string())['token']
"""
Once you have a token, you can pass that in the Authorization header
You can store this in a cache and throw away the user/password
This is just an example query. See http://developer.github.com/v3/
for more about the url structure
"""
resource = Resource('https://api.github.com/user/repos', pool=pool)
headers = {'Content-Type' : 'application/json' }
headers['Authorization'] = 'token %s' % token
response = resource.get(headers = headers)
repos = json.loads(response.body_string())
</pre>
<br/>
There's not any magic in this code, but it took a couple of reads to wade past all of the OAuth talk in github's docs and realize that for a simple browserless tool, you can avoid using OAuth libraries altogether and still not have to store a hard-coded password.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com2tag:blogger.com,1999:blog-2172949199202904007.post-67120542153788525832012-04-26T08:55:00.000-07:002012-04-26T09:53:10.158-07:00Greenhopper, Jira, and RESTOne of the somewhat frustrating problems I'm dealing with in Greenhopper is that I want the ability to treat a linked issue like a subtask, but without all the restrictions of a subtask. Subtasks have at least three limitations that get in my way:<br />
<ol>
<li>They must be in the same project as their parent</li>
<li>They must have the same permissions (issue-level security) as their parent</li>
<li>They must be of an issue type that is flagged as a "subtask" type, so for example, a "Feature" cannot be a subtask of a "Story" unless you create a separate "Feature (subtask)" issues type.</li>
</ol>
Issue #1 is probably the most frustrating, because product management and the exec team at a software company think in terms of high-level features or use cases for which the implementation will often cross project boundaries. (An example at Eucalyptus is that a new use case may require changes to both Eucalyptus and Euca2ools.)<br />
<br />
The Greenhopper UI operates mostly via a REST API, and so far this API is not well documented. Last night I got around this lack of documentation by using <a href="http://mitmproxy.org/" target="_blank">mitmproxy</a> to monitor calls while moving issues up and down the planning page in Greenhopper's Rapid Board. Then I added a simple rest client class to <a href="https://bitbucket.org/agrimm/jiranemo" target="_blank">jiranemo</a> based on <a href="http://pypi.python.org/pypi/restkit" target="_blank">restkit</a>. I made two helper functions: one to get the rest representation of an issue, and another to change the rank of an issue in Greenhopper. My script looks like this:<br />
<br />
<pre class="brush: python">
#!/usr/bin/python
import sys
import pyjira
from jiranemo import jiracfg
# Set the exception hook to enter a debugger on
# uncaught exceptions
from jiranemo.lib import util
sys.excepthook = util.genExcepthook(debug=True,
debugCtrlC=True)
# Read ${HOME}/.jirarc, and set up clients and auth caches.
cfg = jiracfg.JiraConfiguration(readConfigFiles=True)
authorizer = pyjira.auth.CachingInteractiveAuthorizer(cfg.authCache)
ccAuthorizer = pyjira.auth.CookieCachingInteractiveAuthorizer(cfg.cookieCache)
client = pyjira.JiraClient(cfg.wsdl,
(cfg.user, cfg.password),
authorizer=authorizer,
webAuthorizer=ccAuthorizer)
# Do a simple JQL query via the SOAP client, return 20 results
issues = client.client.getIssuesFromJqlSearch(
'''project = "system testing 2" order by Rank DESC''', 20)
for x in issues:
# Get the REST representation of each issue, because links
# aren't shown in the SOAP representation
rest_issue = client.restclient.get_issue(x.key)
for link in rest_issue['fields']['issuelinks']:
if link['type'].has_key('inward') and \
link['type']['inward'] == "is blocked by":
# Rank the linked issue above this one in Greenhopper
result = client.restclient.gh_rank(link['inwardIssue']['key'],
before=rest_issue['key'])
</pre>
The code could use some error checking, but this is a pretty simple starting point for doing something that Jira and Greenhopper can't do on their own.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com2tag:blogger.com,1999:blog-2172949199202904007.post-77201361040702889212012-04-25T07:46:00.000-07:002012-04-25T07:46:02.350-07:00Resurrecting JiranemoAbout six years ago, <a href="https://github.com/dugan" target="_blank">David Christian</a> developed a <a href="http://www.atlassian.com/software/jira/overview" target="_blank">JIRA</a> CLI called jiranemo (his <a href="http://blogs.conary.com/index.php/dugan/2006/10/03/jiranemo" target="_blank">original blog post</a> is, somewhat surprisingly, still around on the rPath website). After he left rPath, I spent some time updating the code for Jira 4 and adding some minor features, but it's been mostly stagnant for about two years. In the meantime, Jira 5 has been released, and the core dependency of jiranemo, SOAPpy, has been declared dead.<br />
<br />
This month, <a href="http://www.eucalyptus.com/" target="_blank">Eucalyptus</a> started on the migration path from using a combination of <a href="http://bestpractical.com/rt/" target="_blank">RT</a> and Launchpad to using Jira. I'm really excited about the change, and it gave me a chance to pick up the <a href="https://bitbucket.org/agrimm/jiranemo" target="_blank">jiranemo code</a> again. I've now converted it from SOAPpy to <a href="https://fedorahosted.org/suds/" target="_blank">suds</a>, and on Monday I used it to import 2000 issues from RT into jira (stay tuned for details on that becoming a publicly-accessible system). I had database access to RT, but all of the interaction with jira was done through the <a href="https://developer.atlassian.com/display/JIRADEV/Creating+a+JIRA+SOAP+Client" target="_blank">SOAP API</a>. ( I realize they now also have a <a href="https://developer.atlassian.com/display/JIRADEV/JIRA+REST+APIs" target="_blank">REST API</a>, which looks awesome, but I already had the code for using SOAP ).<br />
<br />
I should also note that before I took on this work, I looked at <a href="http://toolsmiths.blogspot.com/" target="_blank">Matt Doar</a>'s <a href="https://studio.plugins.atlassian.com/browse/JCLIMD" target="_blank">python-based CLI</a>, which worked well for single commands (and was a reference for some of my jiranemo updates), but it didn't have a library interface, and it seemed very inefficient to keep spawning new python processes for thousands of commands. Jiranemo's separation of the command-line option handling and config file parsing from the client library and helper functions make it fantastic for integrating into more complex python apps.<br />
<br />
I expect that the next phase of development for jiranemo will be a gradual migration toward the REST APIs. If this code is useful to you and you'd like to contribute to this effort, feel free to fork <a href="https://bitbucket.org/agrimm/jiranemo" target="_blank">my bitbucket repo</a> and send me pull requests.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com0tag:blogger.com,1999:blog-2172949199202904007.post-46165873528448820522012-03-08T19:30:00.000-08:002012-05-07T12:45:59.789-07:00An Online Identity CrisisYesterday I was working with some folks in #fedora-java, and after confusion over my IRC versus FAS nicks, someone asked me how many different nicks I had. The answer, unfortunately, is "at least four." I realized a couple of years ago that I had created a problem for myself as far as online identities go. I was never one to go signing up for new services to reserve my nick early, and I've not chosen particularly unique nicks. So, this is me, currently:<br />
<ul>
<li><b>mull</b> - an IRC-only nick that dates back to my college days, maybe 1999-ish, when folks in the #aalug channel were trying out new nicks daily for a while. I have no idea why this stuck, and I've never used it anywhere else. <b>UPDATE:</b> I've now claimed "agrimm" on IRC, so you won't be seeing "mull" anymore.</li>
<br />
<li><b>arg</b> - I made a particularly huge mistake when I chose my initials for my FAS
(Fedora Account System) ID, even though there was very little
possibility I'd be able to use that elsewhere. I suspect that a FAS account is one of the less trivial ones to change, too, so I'm probably stuck with this being a one-off.</li>
<br />
<li><b>agrimm</b> - the obvious first initial + last name choice, more easily obtained than initials, but still not universally unique. I use it for email addresses and not much else, and I get a *lot* of misdirected email for people who share my last name.</li>
<br />
<li><b>a13m</b> - I started using this for twitter and some other random things when arg and agrimm were taken. In case it's not obvious, this nick derives from my full name, in the style of i18n, a11y, etc. While very short and almost never taken by anyone else, the relation to my name is subtle enough that most people don't make the connection.</li>
</ul>
I've clearly been overthinking this, but I really would like to converge on nicks which are clearly related (for example, the guy who has "arg" on FreeNode is "argv0" on Twitter... clever). Any suggestions?Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com1tag:blogger.com,1999:blog-2172949199202904007.post-75269270718788999412012-03-04T17:14:00.000-08:002012-03-04T17:14:26.782-08:00Beware of RHEL 6 / CentOS 6 kernels in Xen guestsA couple of weeks ago, I made my first attempt at creating a multi-hypervisor CentOS 6 image for Eucalyptus for demo purposes. I was pretty sure I had the image creation thing down to a science with ami-creator, but it seems there's always room for error. While I had all the correct drivers in the initrd (usually that's all that really matters), it turns out there's a <a href="https://bugzilla.redhat.com/show_bug.cgi?id=729586" target="_blank">kernel bug affecting device naming on Xen</a>, and since ami-creator currently uses device names rather than UUIDs or labels, my image failed to boot. For those who don't want to read the whole bug, it's stated concisely by Kevin Stange:<br />
<br />
<blockquote class="tr_bq">
<pre class="bz_comment_text" id="comment_text_4">There was effective breakage between kernel-2.6.32-71.29.1.el6 and
kernel-2.6.32-131.el6. When going from 6.0 to 6.1, the result is that if your
Xen domain configuration file specified sda1 as a device name, it was
previously renamed to xvda1. After 2.6.32-131.el6, the device is named xvde1
instead (because the names xvda - xvdd are reserved for hda - hdd device
remapping).
In situations where the configuration file explicitly lists "xvda1" or uses
"hda1", "xvda1" continues to work. </pre>
</blockquote>
<br />
So it seems that there are multiple workarounds to the problem, and it will be fixed in the 6.3 kernel, which is all good. However, I have to say that it's finally made me understand why some of my coworkers prefer their <a href="https://projects.eucalyptus.com/redmine/projects/kernel" target="_blank">"single kernel"</a> project, which aims to provide one kernel / ramdisk which can properly boot several distros on several hypervisors. I'm still partial to running the distro-provided kernel whenever possible, but having a known-good fallback that will at least be able to access the root filesystem & network is nice, so thanks to the Eucalyptus Support / IT team for working on that.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com1tag:blogger.com,1999:blog-2172949199202904007.post-8203003620904242672012-02-02T11:05:00.000-08:002012-02-02T11:05:05.075-08:00Anaconda to the RescueI've always been a fan of the flexibility of anaconda and kickstart not just for installing systems, but also for rescuing a system when something goes horribly wrong. Yesterday I updated a remote test system from Fedora 16 to Rawhide, and I found myself with no network access to the machine due to a <a href="https://bugzilla.redhat.com/show_bug.cgi?id=786937" target="_blank">firmware issue</a>. The system has DRAC 6 express, so I can reset the system and force a pxe boot, but I can't see or interact with the console when it boots. Recent Fedora releases have a great way to rescue a system in this state. First, you set up a kickstart file for the rescue (probably only the first two lines are needed, but I did not test with fewer lines than this):<br />
<br />
<pre>rescue --nomount
sshpw --username=root sekrit --plaintext
url --url http://mirror.eucalyptus/fedora/releases/16/Fedora/x86_64/os/
lang en_US.UTF-8
firewall --enabled --port=22:tcp</pre>
<br />
Then set up these boot options in your PXE configuration:<br />
<br />
<pre>ks=http://yourWebServer/ks/fedora-16-rescue.cfg ksdevice=link keymap=us lang=en_US sshd</pre>
<br />
This works just like rescue mode always has, except you don't need console access. Very cool.<br />
<br />
I'm sure this feature isn't news to a lot of Fedora users, but sometimes cool new features like this sneak into a Fedora release and not everyone realizes it, so it seemed to be worth a quick blog.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com1tag:blogger.com,1999:blog-2172949199202904007.post-79890625799862666362012-01-31T14:32:00.000-08:002012-01-31T14:32:26.352-08:00Image creation, part deuxMy last blog post was a long and quite hackish procedure for running a Fedora install on a live instance in a Eucalyptus 3 cloud... and now I'm going to show you the easier way to build an image. I spent some time kicking around <a href="https://github.com/katzj/ami-creator" target="_blank">ami-creator</a>, and I only ran into a few small issues. I've <a href="https://github.com/eucalyptus/ami-creator" target="_blank">forked it on github</a> and committed the necessary changes. There is a sample kickstart file in the source tree. Installation is a snap (sorry for not having it in rpm form, but that wasn't the goal of the day):<br />
<ul>
<li>easy_install ez_setup</li>
<li>git clone https://github.com/eucalyptus/ami-creator</li>
<li>cd ami-creator</li>
<li>python setup.py build</li>
<li>python setup.py install</li>
<li>mkdir ~/f16-image</li>
<li>cp ks-fedora-16.cfg ~/f16-image/ </li>
<li>cd ~/f16-image</li>
<li>optionally, go modify the kickstart file to point to your mirror, add the packages you want, change the disk size, etc. </li>
<li>ami-creator -c ks-fedora16.cfg -n f16test -v -e</li>
</ul>
When the process completes, you'll have a few new files in the current directory:<br />
<ul>
<li>f16test.img</li>
<li>initramfs-3.2.2-1.fc16.x86_64.img</li>
<li>initrd-plymouth.img</li>
<li>vmlinuz-3.2.2-1.fc16.x86_64</li>
</ul>
You can ignore initrd-plymouth.img. Just go through the normal steps of bundle, upload, and register for each of the other three files, and you should have a working Fedora EMI. It can't get much simpler than that. Thanks to <a href="http://velohacker.com/" target="_blank">Jeremy Katz</a> for starting the ami-creator project. I hope that someday we'll see this rolled into the <a href="http://git.fedorahosted.org/git/livecd" target="_blank">live image tools</a> project where it belongs.<br />
<br />
<br />
<br />Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com2tag:blogger.com,1999:blog-2172949199202904007.post-52480709417310100542012-01-31T09:02:00.000-08:002012-01-31T09:02:58.930-08:00Image creation in the cloudThis post is the result of a challenge given to me by Seth Vidal, which showed up in <a href="http://skvidal.wordpress.com/2012/01/29/euca-thoughts/" target="_blank">his weekend blog post</a>. He was musing about whether it's possible to actually do a kickstart, or even an interactive install, in a cloud instance. I have to put some disclaimers around this post, because I am _not_ advocating this approach, and I'm going to show you a feature of Eucalyptus 3 that could void your warranty if used in anger. As my friend Michael likes to say, if you break it, you get to keep both pieces.<br />
<br />
What we hashed out on Friday was that, in order to be able to kickstart inside an instance, you have to be able to pass boot parameters. In Eucalyptus 2, the only real way to do this was by patching the node controller with something similar to the <a href="https://geni-orca.renci.org/trac/wiki/NEuca-overview" target="_blank">NEuca</a> patches. In Eucalyptus 3, we've implemented a sort of "escape hatch" called nc-hooks to allow folks to customize behaviors at instance definition and launch time. There's an example shell script in /etc/eucalyptus/nc-hooks/ which shows how you might write your own hooks.<br />
<br />
Knowing that the nc-hooks feature existed, I had to think about exactly how to pass boot parameters and get them into libvirt.xml before instance launch. Passing them via userData was the obvious choice. I came up with a couple of xslt files and this script to make the magic happen:<br />
<br />
<br />
<pre>#!/bin/sh
event=$1
euca_scripts=/home/eucalyptus/scripts
inst_home=$3
rewrite_libvirt_xml() {
# Get only the value of the "bootparams=..." line from userData
BP=$( xsltproc $euca_scripts/get-user-data.xsl $inst_home/instance.xml \
| base64 -d \
| sed -r "/bootparams=/!d; s/^.*bootparams=(.*)/\1/" || exit 1 )
# Substitute the value of $BP into the stylesheet
sed -e "s!@@BOOTPARAMS@@!$BP!" < $euca_scripts/insert-boot-params.xsl \
> $inst_home/insert-boot-params.xsl || exit 2
# Rewrite and replace libvirt.xml for this instance
xsltproc $inst_home/insert-boot-params.xsl $inst_home/libvirt.xml \
> $inst_home/libvirt.xml.new || exit 3
cp $inst_home/libvirt.xml $inst_home/libvirt.xml.orig
mv -f $inst_home/libvirt.xml.new $inst_home/libvirt.xml
}
case "$event" in
euca-nc-pre-boot)
rewrite_libvirt_xml
exit 0
;;
*)
exit 0
;;
esac
</pre>
I don't have a vast amount of experience when it comes to xml processing, so forgive the horror of these stylesheets. The first one, get-usr-data.xsl, is quite simple:<br />
<br />
<pre><?xml version="1.0" encoding="UTF-8"?>
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output encoding="UTF-8" indent="yes" method="text"/>
<xsl:template match="/instance">
<xsl:value-of select="/instance/userData"/>
</xsl:template>
</xsl:transform>
</pre>
<br />
The second is a little stranger, and was done with some help from StackOverflow:<br />
<br />
<pre><?xml version="1.0" encoding="UTF-8"?>
<xsl:transform xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output encoding="UTF-8" omit-xml-declaration="yes" indent="yes" method="xml"/>
<xsl:template match='node()|@*'>
<xsl:copy>
<xsl:apply-templates select='node()|@*'/>
</xsl:copy>
</xsl:template>
<xsl:template match="cmdline">
<cmdline>@@BOOTPARAMS@@</cmdline>
</xsl:template>
</xsl:transform></pre>
<br />
So with these files in place, I now need to configure an installer kernel and ramdisk. These come from the /fedora/releases/16/Fedora/x86_64/os/images/pxeboot/ directory of your favorite Fedora mirror site. The kernel and ramdisk registration process is the usual:<br />
<br />
<ul>
<li> euca-bundle-image --kernel true -i vmlinz</li>
<li>euca-upload-bundle -b f16 -m /tmp/vmlinuz.manifest.xml</li>
<li>euca-register f16/vmlinuz.manifest.xml</li>
<li>euca-bundle-image --ramdisk true -i initrd.img</li>
<li>euca-upload-bundle -b f16 -m /tmp/initrd.img.manifest.xml</li>
<li>euca-register f16/initrd.img.manifest.xml</li>
</ul>
I don't really *need* a disk image here, but an EMI cannot be registered without one, so I fake it:<br />
<ul>
<li> dd if=/dev/zero of=fake-emi.img bs=1k count=10000</li>
<li>mke2fs fake-emi.img</li>
<li>euca-bundle-image -i fake-emi.img</li>
<li>euca-upload-bundle -b f16 -m /tmp/fake-emi.img.manifest.xml</li>
<li>euca-register --kernel eki-EA183EA8 --ramdisk eri-6ED23EF2 f16/fake-emi.img.manifest.xml</li>
</ul>
Note that even if you aren't trying to do key injection, the disk image needs to have an ext2-compatible filesystem on it.<br />
<br />
Next, I need a volume to install into:<br />
<ul>
<li>euca-create-volume -s 10 -z PARTI00 </li>
</ul>
Before I boot the instance, here's where I have to be honest with you readers: I make a lot of mistakes when I test things like this. Typos, logic errors, you name it. So for debugging purposes, I uncomment this line in /etc/eucalyptus/libvirt.xsl:<br />
<br />
<pre><graphics type='vnc' port='-1' autoport='yes' keymap='en-us' listen='0.0.0.0'/>
</pre>
<br />
You definitely should not have this line uncommented for normal use, as it will allocate a port for vnc for every instance you launch, and without some extra configuration, it doesn't even require a password to connect. For quick debugging on a safe network, though, it's a good way to see what's going wrong during the boot process.<br />
<br />
Now to launch my installer instance:<br />
<br />
<pre>euca-run-instances -t m1.xlarge \
-d "bootparams=ksdevice=link ip=dhcp vnc keymap=us lang=en_US console=ttyS0" \
emi-BA8F405E
</pre>
<br />
This boots into an interactive install, which listens for vnc connections. Note that due to the size of the initrd, this instance needs a significant amount of RAM; I used 2GB, but 1GB would have worked. Before proceeding, I attach the volume (which I could have done via block device mapping):<br />
<br />
euca-attach-volume -i i-447E3E89 -d sdd vol-14AE3F68<br />
<br />
<br />
I check euca-describe-instances for the instance's IP address, connect to it with a vnc client, and proceed with the install. Once the install completes, I detach the volume and terminate the instance:<br />
<br />
<ul>
<li>euca-detach-volume vol-14AE3F68</li>
<li>euca-terminate-instances i-447E3E89</li>
</ul>
<br />
Finally, I convert the volume to a snapshot and register it:<br />
<ul>
<li>euca-create-snapshot vol-14AE3F68</li>
<li>euca-register -n f16-test -s snap-2CBB42D9</li>
</ul>
<br />
I boot an instance of my new EMI, and ... it fails to have a network. There were multiple problems with the networking configuration:<br />
<ol>
<li>The MAC address is hard-coded.</li>
<li> The device name has changed from eth0 to eth1 (maybe related to #1)</li>
<li>The NIC is configured to be controlled by NetworkManager</li>
</ol>
This is when I'm happy to have a vnc connection provided at the libvirt layer to debug the instance. A quick setup of ifcfg-eth1 and a restart of the network gives me connectivity, and I'm up and running with a Fedora 16 instance installed entirely in the cloud.<br />
<br />
The whole process took me about an hour or so this morning (not counting writing the xsl and shell script yesterday), and I imagine that the process would be much faster for subsequent attempts, and even faster when a kickstart is used. Still, I'm not convinced that an approach like this has significant value over something like <a href="http://boxgrinder.org/" target="_blank">BoxGrinder</a> or <a href="http://velohacker.com/fedora-notes/announcing-ami-creator/" target="_blank">ami-creator</a>. Let the debate begin! :-)Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com2tag:blogger.com,1999:blog-2172949199202904007.post-89383681791243334272012-01-19T17:29:00.000-08:002012-10-08T12:22:18.578-07:00Configuring Eucalyptus 3-develIn my <a href="http://agrimmsreality.blogspot.com/2012/01/building-eucalyptus-3-devel.html">last entry</a>, I explained how to checkout eucalyptus 3-devel and build it from source on Fedora 16. This entry will explain how to follow that process with configuration and initialization of a single node cloud.<br />
<br />
1) Configure environment variables.<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">export EUCALYPTUS=/opt/eucalyptus</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">export PATH=$PATH:$EUCALYPTUS/usr/sbin</span></span><br />
<br />
2) Configure eucalyptus.conf -- Since this is a single node install on a network with DHCP, I am using SYSTEM mode for networking, which is the default.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">EUCALYPTUS="/opt/eucalyptus"</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">HYPERVISOR="kvm"</span></div>
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">USE_VIRTIO_DISK="1"</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">USE_VIRTIO_NET="1</span><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">"</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">INSTANCE_PATH="/opt/eucalyptus/instances"</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">VNET_BRIDGE="br0"</span></span><br />
<br />
<br />
3) Set up proper file and directory permissions in the installed tree:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">su -c "euca_conf --setup"</span></span><br />
<br />
4) Initialize the database:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">euca_conf --initialize</span></span><br />
<br />
5) Create a bridge device and associate your primary NIC (this is specific to SYSTEM mode):<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/etc/sysconfig/network-scripts/ifcfg-br0:<br />DEVICE=br0<br />TYPE=Bridge<br />BOOTPROTO=dhcp<br />ONBOOT=yes<br />DELAY=0<br />NM_CONTROLLED=no</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;"></span></span><br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">/etc/sysconfig/network-scripts/ifcfg-em1:</span></span><br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">DEVICE="em1"<br />ONBOOT=yes<br />BRIDGE=br0<br />NM_CONTROLLED=no</span></span>then restart your network<br />
<br />
6) <b>UPDATE: </b>Start the CLC <b>before</b> getting credentials.<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">su -c "/opt/eucalyptus/etc/init.d/eucalyptus-cloud start"</span></span><br />
<br />
7) Get credentials and source them:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">euca_conf --get-credentials admin.zip</span></span><br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">unzip admin.zip</span></div>
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">source eucarc</span></span><br />
<br />
8) Start the cloud components and register services:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">euca_conf --register-walrus -H <hostname> -C walrus -P walrus</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">euca_conf --register-sc -H <hostname> -C SC_251 -P PARTI00</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">euca_conf --register-cluster -H <hostname> -C CC_251 -P PARTI00</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">su -c "/opt/eucalyptus/etc/init.d/eucalyptus-cc start"</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">euca_conf --register-nodes <hostname></span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">su -c "/opt/eucalyptus/etc/init.d/eucalyptus-nc start" </span></div>
<br />
At this point, you should have a running cloud. To verify the components:<br />
<br />
euca-describe-walruses ; euca-describe-storage-controllers ; euca-describe-clusters<br />
<br />
You should see something like:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">WALRUS walrus walrus 192.168.51.251 ENABLED {}</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">STORAGECONTROLLER PARTI00 SC_251 192.168.51.251 ENABLED {}</span><br style="font-family: "Courier New",Courier,monospace;" /><span style="font-family: "Courier New",Courier,monospace;">CLUSTER PARTI00 CC_251 192.168.51.251 ENABLED {}</span></span><br />
<br />
And to ensure that the node controller is advertising resources:<br />
<br />
euca-describe-availability-zones verbose<br />
<br />
which shows: <br />
<br />
AVAILABILITYZONE PARTI00 192.168.51.251 arn:euca:eucalyptus:PARTI00:cluster:CC_251/<br />
AVAILABILITYZONE |- vm types free / max cpu ram disk<br />
AVAILABILITYZONE |- m1.small 0004 / 0004 1 128 2<br />
AVAILABILITYZONE |- c1.medium 0002 / 0002 1 256 5<br />
AVAILABILITYZONE |- m1.large 0001 / 0001 2 512 10<br />
AVAILABILITYZONE |- m1.xlarge 0000 / 0000 2 1024 20<br />
AVAILABILITYZONE |- c1.xlarge 0000 / 0000 4 2048 20<br />
<br />
That's all for my second post. Comments and corrections welcome. See you on #eucalyptus !Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com2tag:blogger.com,1999:blog-2172949199202904007.post-84235313869827392412012-01-19T11:03:00.000-08:002012-06-15T07:41:14.417-07:00Building Eucalyptus 3-develAs some readers may already be aware, Eucalyptus has started publishing code from the Eucalyptus 3 development branch on launchpad. The build process has a fairly sizable set of dependencies, so I'd like to give a quick example of how I built and installed this code on a Fedora 16 system. I started with a minimal x86_64 install.<br />
<br />
1) Add Eucalyptus' yum repository which contains dependencies for the source build:<span style="font-size: x-small;"> </span><br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">[euca-deps]</span><span style="font-size: x-small;"> </span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">name=euca-deps</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">baseurl=http://downloads.eucalyptus.com/devel/packages/3-devel/fedora/16/x86_64/</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">gpgcheck=1</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">enabled=1</span></div>
<br />
<pre class="code"></pre>
2) Download the GPG key for verifying packages, and add it to your rpm database:<br />
<br />
<span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">rpm --import http://downloads.eucalyptus.com/devel/gpg-keys/9d7b073c-eucalyptus-nightly-release-key.pub</span></span><br />
<br />
3) Install dependencies. For simplicity, this list includes build and runtime deps for all components:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">yum install axis2c rampartc axis2c-devel rampartc-devel python-boto \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">euca2ools libvirt-devel openssl-devel gcc java-1.6.0-openjdk-devel ant \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">curl-devel libxslt-devel apache-commons-logging xalan-j2-xsltc wsdl4j \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">backport-util-concurrent httpd postgresql-server libvirt PyGreSQL make \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">openssh-clients scsi-target-utils qemu-kvm axis2-codegen axis2-adb-codegen</span></div>
<br />
4) <strike>Install grub 1.</strike> <b>UPDATE:</b> This is no longer required in 3.1.<br />
<br />
This package is obsoleted by grub2, so it cannot be installed by yum, but it has no conflicting files, so installing it outside of the rpm database is safe:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">yum install yum-utils</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;"> yumdownloader grub</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">cd /; rpm2cpio /root/grub-0.97-*.rpm | cpio -id</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">cp /usr/share/grub/x86_64-redhat/* /boot/grub/</span></div>
<br />
5) Create a user named eucalyptus on your system.<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">useradd -G kvm eucalyptus</span></div>
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">passwd eucalyptus</span><br />
<br />
6) Disable iptables (eucalyptus must be allowed to control iptables for dynamic routing).<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">systemctl disable iptables.service</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">systemctl stop iptables.service</span></div>
<br />
7) Increase shmmax on the system:<span style="font-size: x-small;"> </span><br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">SHMMAX=$(( 48 * 1024 * 1024 ))</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">echo $SHMMAX > /proc/sys/kernel/shmmax</span></div>
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">e</span><span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">cho "kernel.shmmax = $SHMMAX" > /etc/sysctl.d/euca_shmmax</span><br />
<br />
8) Modify /usr/lib64/axis2c/bin/tools/wsdl2c/WSDL2C.sh -- erase the existing lines and add these:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">java -classpath $(build-classpath axis2/codegen axis2/kernel axis2/adb \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;"> axis2/adb-codegen wsdl4j commons-logging xalan-j2 xsltc \</span></div>
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;"> backport-util-concurrent ws-commons-XmlSchema ws-commons-neethi \</span></div>
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;"> ws-commons-axiom annogen ) org.apache.axis2.wsdl.WSDL2C $*</span><br />
<br />
9) Log in as the eucalyptus user to checkout and build the code<br />
<br />
10) <strike>Check out the code from bzr: <span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">bzr branch lp:eucalyptus && cd eucalyptus</span></strike><br />
<b><span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">UPDATE: </span></span></b>Check out the code from GitHub: git clone https://github.com/eucalyptus/eucalyptus.git<br />
<br />
11) Configure eucalyptus. I recommend running "./configure --help" and reading over the options, but this configuration should work:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">./configure --with-axis2c=/usr/lib64/axis2c/ \</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;"> --with-apache2-module-dir=/usr/lib64/httpd/modules/</span> <br />
<br />
12) Modify ./clc/modules/postgresql/conf/scripts/setup_db.groovy :<br />
<br />
Change PG_BIN on line 97 to "/usr/bin/pg_ctl"<br />
Change PG_INITDB on line 103 to "/usr/bin/initdb"<br />
<br />
13) Run <span style="font-size: x-small;"><span style="font-family: "Courier New",Courier,monospace;">make</span></span><br />
<br />
Then as root, do the following:<br />
<br />
14) Run <span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">make install</span><br />
<br />
NOTE: there's an issue here which causes some files in the source tree to be root-owned after this step, so you may want to run "find . | xargs chown eucalyptus" to fix this. Otherwise, you may see "permission denied" errors the next time you run "make" or "make distclean".<br />
<br />
15) Copy PolicyKit configuration for libvirt into place:<br />
<br />
<div style="font-family: "Courier New",Courier,monospace;">
<span style="font-size: x-small;">mkdir -p /var/lib/polkit-1/localauthority/10-vendor.d</span></div>
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">cp -p tools/eucalyptus-nc-libvirt.pkla \</span><br />
<span style="font-family: "Courier New",Courier,monospace; font-size: x-small;"> /var/lib/polkit-1/localauthority/10-vendor.d/eucalyptus-nc-libvirt.pkla</span><br />
<br />
16) Restart libvirtd: <span style="font-family: "Courier New",Courier,monospace; font-size: x-small;">systemctl restart libvirtd.service</span><br />
<br />
I'll write up another post detailing configuration and initialization steps. For those of you who have used eucalyptus before, you will find that the eucalyptus.conf file is mostly unchanged from 2.0.x. The database initialization and component registration steps differ slightly, though. Stay tuned.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com4tag:blogger.com,1999:blog-2172949199202904007.post-17617961110934720932011-12-29T08:11:00.000-08:002011-12-29T08:55:59.906-08:00NetworkManager CLIIn the past few years, there's been a proliferation of wireless access points, and the NetworkManager applet does not deal with this well. I have seen cases where there are so many access points found during a scan that the one I want doesn't make the list. I suppose it's understandable if it's one I've never connected to, and if the signal isn't one of the strongest available. If I've made clear that it's one of my preferred connections, though, it should always be found. Rather than stew over this brokenness, I was determined to find a way around it. NetworkManager CLI to the rescue!<br />
<br />
I'm not deeply familiar with the NetworkManager CLI, but only a couple of commands are needed.<br />
"nmcli con" shows a list of connections with names and UUIDs. Sample output:<br />
<br />
<pre>NAME UUID TYPE TIMESTAMP-REAL
Auto GloblaSuiteWireless 1aa7870c-0408-4c14-a42a-2e706cabed84 802-11-wireless Tue 28 Jun 2011 12:43:20 PM EDT
Auto FLYSBA 37308cf9-b520-4754-b382-8aec39fea79e 802-11-wireless Fri 27 May 2011 11:19:16 PM EDT
euca 42696e07-1b3a-49c8-9fd3-4483d9211285 vpn Thu 29 Dec 2011 11:34:19 AM EST
</pre>
<br />
If you travel much, you probably have a huge list of these, so grep is your friend. Once you've found the connection you wish to activate, run this with the appropriate uuid string:<br />
<br />
<pre>sudo nmcli con up uuid 37308cf9-b520-4754-b382-8aec39fea79e
</pre>
<br />
I've set up script aliases for my favorite connections, so I don't have to search through the list each time. Note that VPN connections can be activated in this way as well.<br />
<br />
As always, I hope this little nugget of information saves someone a bit of time someday. It's certainly made my life easier.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com1tag:blogger.com,1999:blog-2172949199202904007.post-15144095584545368052011-12-28T12:13:00.000-08:002011-12-28T12:13:19.514-08:00Old School Audio PlaybackOne of the things that can be frustrating as an "old school" Linux user is that, with each upgrade to a new distro / release, familiar lightweight tools for everyday tasks fall out of fashion, and flashier new GUIs take their place. Often, the older tools are still available, but they may not work out-of-the-box as they once did. When I find something like this, I think it's worthwhile to document it publicly. And while what I'm documenting may be obvious to some people, my perspective is that if it took me more than 30 seconds to figure it out, someone else is going to search the web for it eventually.<br />
<br />
I have an old workstation running Fedora 16 in text mode (partly because I don't need a GUI, and partly because the nouveau graphics driver doesn't behave well on this system). Today I wanted to play a CD and went looking for a CLI to do so... nothing? Okay, I was going to rip it eventually, so cdparanoia + vorbis-tools (oggenc) got that job done. Next, I tried ogg123 to play the converted files ... no sound. There are probably some hoops I could have jumped through to get PulseAudio working, but I found that after a simple "alsactl init", ogg123 worked just fine. I don't recall having to explicitly initialize the sound card like this in recent years, but it's good to know how simple it is. <br />
<br />
It's worth noting that ogg123 uses about 5 MB of RAM, and about 1% of my CPU (maybe less) during playback. Very nice.Andy Grimmhttp://www.blogger.com/profile/13824322599959355928noreply@blogger.com1