Edmund Haselwanter's Blog Feed

HTTP Basic Auth for Mysql password and old_password in Rails

Mon, 02 Nov 2009 14:52:00 GMT

I recently had to develop a small application for a client. The goal was to add a feature to grown Java web application.

As the software was maintained for several years there are users with old an new MySql passwords in the system.

To authenticate the user I have chosen authenticate_or_request_with_http_basic

As I just needed handful of actions all of them go in one controller:

   1  class MyController < ApplicationController
   2  
   3    before_filter :authenticate
   4  
   5    def index
   6     # [ ... ]
   7    end
   8  
   9    protected
  10  
  11    def authenticate
  12      authenticate_or_request_with_http_basic do |id, password|
  13        @user = User.find_by_name(id)
  14        login_ok = @user.authenticate(password)
  15        login_ok
  16      end
  17    end
  18  
  19  end

To check if the user can be authenticated:

   1  class User < ActiveRecord::Base
   2  
   3    def authenticate(password)
   4      user = User.find_by_sql(["select * from user where user_pass=password('%s') \
   5                                OR user_pass=old_password('%s')",password,password])
   6      (user.length > 0)?true:false
   7    end
   8  
   9  
  10  end
  11

Its a very simple approach.

What do you think?

Technorati Tags: http basic auth, mysql, rails

chef from opscode and the power of ruby

Wed, 14 Oct 2009 09:52:00 GMT

Recent releases of chef from opscode provide a new resource: ruby_block

This is really great news as this allows for interesting problem solving.

Consider you want to do a very special HTTP request. E.g. a GET to localhost but with custom “Host” header information.

Install a Ruby Http Client

First we have to install a Ruby http client library. I’ll use curb in this example. It builds on top of libcurl so lets install that first.

   1  curl_devel = package "curl-devel" do
   2    action :install
   3  end
   4  
   5  curl_devel.run_action(:install)

With this code we tell chef to immediately execute this code. Otherwise we would have to face the problem that we just fullfill the chefrun post-condition that we have curl-devel installed. Next we have to install the curb gem.

   1  curb_gem = gem_package "curb" do
   2    action :install
   3  end
   4  
   5  curb_gem.run_action(:install)

as you see this is done the same way. Next we want to use curb in THIS chef-run. So we have tell chef the update of Gem, followed by requiring curb

   1  Gem.clear_paths
   2  
   3  require 'curb'

Ruby Code in Action

Now we are ready to take action.

    2     3     4     5     6     7     8     9    10    11    12    13    14    15    16    17    18    19    20

1 ruby_block "do-http-request-with-cutom-header" do block do timeout = 600 host = "localhost:8080" real_host = "my.realhost.com" Chef::Log.info "call get on #{host}, maximal request time: #{timeout} seconds" c = Curl::Easy.new("http://#{host}/maintenance/do_something") do |curl| curl.headers['Host'] = real_host curl.verbose = true curl.timeout = timeout end c.perform if c.response_code == 200 Chef::Log.info "GET success! response was:#{c.body_str}" else Chef::Log.error "GET FAILED. request response was HTTP #{c.response_code}, body: #{c.body_str}" end end action :create end

Great. Isn’t it? :-)

Technorati Tags: chef, opschef, opscode, opscookbook, sysadmin

webalizer cookbook and definition for chef from opscode

Tue, 13 Oct 2009 15:42:00 GMT

Install and run the webalizer with Chef from Opscode

http://github.com/ehaselwanter/cookbooks/tree/COOK-195/webalizer

As DRY as possible ;-)

What it is for

Simply put it derives the log file name from its name and takes a server name for where to access the output of webalizer.

How to use it

The simplest use case is

   1  
   2  webalizer "your.example.com" do
   3    server_name "stats.example.com"
   4  end

This looks for a access log file in

#{node[:apache][:log_dir]}/your.example.com-access.log

und puts the webalizer results (/etc/cron.daily/00-weablizer must be run either manually or from cron)

/var/www/usage/your.example.com

accessable through

http://stats.example.com/your.example.com

options are

   1  
   2  webalizer "your.example.com" do
   3    server_name "stats.example.com"     # custom server name
   4    conf_template "webalizer.conf.erb"  # custom webalizer config
   5    site_template  "webalizer.site.erb"   #custom webalizer apache file
   6    cron_schedule :daily  # node[:webalizer][:cron_base_prefix]}#{params[:cron_schedule].to_s must be a cron directory
   7    log_type :clf # :clf :ftp or :squid
   8    cookbook "webalizer" were the templates reside
   9  end
  10

Technorati Tags: apache2, chef, opschef, opscode, opscookbook, sysadmin

Simple ping.fm extension for radiant

Sun, 11 Oct 2009 15:57:00 GMT

I just created a simple Ping.fm extension for RadiantCMS on github: ping.fm extension

With this you can have updates to your site automatically send details to your
Ping.fm account. All you’ll need to do is provide your account information in the Radiant::Config settings.

API Key http://ping.fm/key/
Application API Key http://ping.fm/developers/

In the first iteration it just posts a link to the page and the title of the page to ping.fm.
If you have the tags extension installed it will add the page meta_tags to the post.

Installation:

   1  gem install pingfm # sudo gem install pingfm
   2  cd /path/to/radiant-installation/
   3  
   4  git clone git://github.com/ehaselwanter/radiant-pingfm-extension.git vendor/extensions/pingfm
   5  
   6  or
   7  
   8  git submodule add git://github.com/ehaselwanter/radiant-pingfm-extension.git vendor/extensions/pingfm
   9  
  10  git submodule init
  11  git submodule update
  12  
  13  rake db:migrate:extensions

Technorati Tags: ping.fm, radiantcms

Glassfish cookbook for chef from opscode

Sun, 11 Oct 2009 14:50:00 GMT

Install and run the Sun Glassfish Application Server from Sun with Chef from Opscode

At the moment its really dumb and just installs Glassfish and runs it:
http://github.com/ehaselwanter/cookbooks/tree/COOK-191/glassfish/

Technorati Tags: chef, Glassfish, java, opschef, opscookbook, Sun

cruise from Thougthworks and ant 1.7.1 on centos 5.2

Tue, 29 Sep 2009 14:03:00 GMT

This is on howto use apache ant 1.7.1 in Cruise 1.2 from Thoughtworks

Since Cruise 1.2 ant ist not bundled with Cruise. So the way to go is install either ant for the cruise user (agent), or provide a system wide ant installation.

fetch ant from apache.org:“http://ant.apache.org/”

   1  cd /opt
   2  wget http://mirror.deri.at/apache/ant/binaries/apache-ant-1.7.1-bin.tar.gz

install it in a suitable place

   1  tar -xzvf apache-ant-1.7.1-bin.tar.gz
   2  ln -s apache-ant-1.7.1 ant

and add the following file

   1  vim /etc/profile.d/ant.sh 
   2  export ANT_HOME=/opt/ant
   3  export PATH=/opt/ant/bin:$PATH

restart the cruise agent to pick up the new settings

   1  vim /etc/profile.d/ant.sh 
   2  export ANT_HOME=/opt/ant
   3  export PATH=/opt/ant/bin:$PATH

verify your new settings:

   1  [cruise@test:~] env |grep ANT
   2  ANT_HOME=/opt/ant
   3  [cruise@test:~] ant -version
   4  Apache Ant version 1.7.1 compiled on June 27 2008

Technorati Tags: ant, centos, cruise, Thougthworks

Extending Chef from Opscode with header params for http_request

Fri, 31 Jul 2009 15:20:00 GMT

Added support for custom header options in chef for the http_request

Opscode Chef Ticket
Resolved on Github

An example using a custom "Host" header property:

   1  
   2  http_request "localhost_with_host_header" do
   3    url "localhost"
   4    message "my request"
   5    headers Hash[:Host => "my-real.hostname.com"]
   6  end

The new parameter is headers which takes a Hash and will add all values.to_s to the request header.
Technorati Tags: http_request, opschef, opscode

Upgraded Page to Radiant 0.8

Sun, 28 Jun 2009 12:52:00 GMT

used extensions

blog
copy_move
gallery
link_roll
paperclipped
reorder
tags
twitter_tags
code
custom_tags
language_redirect
page_preview
ray
settings
twitter
wym_editor_filter

Technorati Tags:

Twitter notification

Sun, 28 Jun 2009 12:44:00 GMT

added Twitter notification to blog posts. Testing with Ping.fm
Technorati Tags:

Howto add Secure Shell Client SSH Key to OpenSSH Server

Wed, 13 May 2009 18:05:00 GMT

You have to convert the key like so:

ssh-keygen -i -f id_dsa_2048_a.pub >>.ssh/authorized_keys

Technorati Tags: ssh

Chef: Cook your own automated environment

Fri, 13 Mar 2009 15:54:00 GMT

There is a new kid on the configuration management block Chef from Opscode. Here is a blog post about Puppet vs. Chef.

Chef is kind of Puppet with integrated iClassify and a rubyish configuration DSL. No wonder, Adam Jacob joined forces on Chef.

You can find it on github

Technorati Tags: chef, environment 2.0, iclassify, opschef, opscode, opscookbook, puppet

Tomcat: Too many open files

Fri, 13 Mar 2009 15:31:00 GMT

the symptom

If tomcat just respondes with HTTP 500 and you find something like

   1  
   2  org.apache.tomcat.jni.Error: Too many open files
   3         at org.apache.tomcat.jni.Socket.accept(Native Method)
   4         at org.apache.tomcat.util.net.AprEndpoint$Acceptor.run(AprEndpoint.java:1110)
   5         at java.lang.Thread.run(Thread.java:619)
   6  Mar 12, 2009 12:01:34 PM org.apache.tomcat.util.net.AprEndpoint$Acceptor run
   7  SEVERE: Socket accept failed

in catalina.out

then you might have too many open file handles.

Too be sure … lsof and ulimit are you friends

use

ulimit -a to see you current configuration:

   1  
   2  
   3  $ ulimit -a
   4  core file size          (blocks, -c) 0
   5  data seg size           (kbytes, -d) unlimited
   6  scheduling priority             (-e) 0
   7  file size               (blocks, -f) unlimited
   8  pending signals                 (-i) 13664
   9  max locked memory       (kbytes, -l) 32
  10  max memory size         (kbytes, -m) unlimited
  11  open files                      (-n) 1024
  12  pipe size            (512 bytes, -p) 8
  13  POSIX message queues     (bytes, -q) 819200
  14  real-time priority              (-r) 0
  15  stack size              (kbytes, -s) 8192
  16  cpu time               (seconds, -t) unlimited
  17  max user processes              (-u) 13664
  18  virtual memory          (kbytes, -v) unlimited
  19  file locks                      (-x) unlimited

open files (-n) 1024

shows there are 1024 possible open file handles per process.

ulimit -n <value> lets you set the value. You might increase it to e.g. 4096. But be sure there is not something in you code which consumes too much file handles. And don’t forget to restart the process after tweaking with ulimit.

to check for the current open file handles use lsof

lsof |grep tomcat shows you all open handles from tomcat (or use grep java)

to get a number to compare with you settings wc comes in handy :-)

   1  
   2  $ lsof |grep tomcat|wc -l
   3  136

Technorati Tags: java, tomcat

Setting up an SSL secured Webserver with CentOS

Mon, 09 Mar 2009 17:02:00 GMT

Here is a very good HOWTO for setting up an SSL secured Webserver with CentOS

ensure there is no ssl.conf for the default VirtualServer in /etc/httpd/conf.d/ or incorporate your changes there.

Technorati Tags: apache, apache2, centos, ssl, virtualhost

Use EBS on EC2 Centos 5.2 and snapshot with cron to S3

Fri, 06 Mar 2009 15:30:00 GMT

Create the volume and assign it to your instance

Attach it to your Centos 5.2 instance

yum install xfsprogs

according to Thread: Debian Lenny and XFS causes kernel panic: Suggested Workaround

mkfs.xfs -l version=1 /dev/sdh

insert into /etc/fstab

/dev/sdh /vol/shared xfs noatime 0 0

mount the volume

mount /vol/shared

Setup ec2-extra-tools

copy over your EC2_CERT and EC2_PRIVATE_KEY

setup /etc/profile.d/ec2.sh

   1  
   2  export EC2_HOME=/home/ec2 
   3  export EC2_CERT=/root/.ec2/cert-xxx.pem
   4  export EC2_PRIVATE_KEY=/root/.ec2/pk-xxx.pem
   5  export AWS_ACCOUNT_NUMBER=
   6  export AWS_ACCESS_KEY_ID=
   7  export AWS_SECRET_ACCESS_KEY=
   8  export PATH=$PATH:/home/ec2/bin/

install http://github.com/cmer/ec2-extra-tools/tree/master

add the binaries to e.g. /home/ec2/bin/

test the setup with:

ec2-xfs-snapshot /dev/sdh

sample output:

   1  

   2  [root@domU-XX-XX-XX-XX-XX-XX:~/] ec2-xfs-snapshot /dev/sdh

   3  device: /dev/sdh

   4  mount point: /vol/shared

   5  ec2-volume-id: vol-67890a

   6  

   7  Freezing XFS volume…

   8  

   9  Creating snapshot…

  10  SNAPSHOT        snap-b99bh1d8   vol-67890a    pending 2009-03-06T14:56:17+0000

  11  Unfreezing XFS volume…

  12  

  13  Done!

Setup cron


crontab -e
SHELL=/bin/bash
PATH=/usr/bin:/bin:/usr/sbin
5 20 * * * source /etc/profile.d/java.sh; source /etc/profile.d/ec2.sh ; \ 
ec2-xfs-snapshot /dev/sdh >> /var/log/ebs_snapshots  2>> /var/log/ebs_snapshots

you need both: java.sh and ec2.sh for the amazon tools.

Technorati Tags: amazon, aws, cloud, cron, ebs, ec2, environment 2.0

Thoughtworks Cruise 1.1 on Centos 5.2 EC2 building repo from github

Sun, 30 Nov 2008 21:53:00 GMT

In this blog post I will show how to launch a Centos 5.2 Amazon ami image, install Cruise 1.1 from Thoughtworks (also known as Cruise Control) and building a Java project, fetched from a github repository.

Start new image with ElasticFox

RightImage CentOS 5.2 Base Image, Version 4.0.2 (32-bit) (ami-1363877a)
Centos 5.2.

conntect to it with ssh

ssh -i $EC2_PAMKEY_FILE rootec2-XX-XX-XX-XX.compute-1.amazonaws.com@

install git (version working with submodules, last with working dependencies from kernel.org)

cd /etc/yum.repos.d/; wget http://kernel.org/pub/software/scm/git/RPMS/git.repo

add

 


[git]
name=Base source git repository
baseurl=http://www.kernel.org/pub/software/scm/git/RPMS/SRPMS
enabled=1
gpgcheck=0

to git.repo

 


yum install yum-utils
yumdownloader --source git
rpm -i git-1.6.0.2-1.fc9.src.rpm 
yum install openssl-devel curl-devel expat-devel xmlto asciidoc
rpmbuild -bb /usr/src/redhat/SPECS/git.spec

… make a coffee brake …

Wrote: /usr/src/redhat/RPMS/i386/git-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-all-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-svn-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-cvs-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-arch-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-email-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-gui-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/gitk-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/perl-Git-1.6.0.2-1.i386.rpm Wrote: /usr/src/redhat/RPMS/i386/git-debuginfo-1.6.0.2-1.i386.rpm cd /usr/src/redhat/RPMS/i386/ yum remove git rpm -i perl-Git-1.6.0.2-1.i386.rpm git-1.6.0.2-1.i386.rpm

git —version git version 1.6.0.2

To be able to use Cruise you must register for a licence key at thougthworks

fetch rpm files

 


wget http://download01.thoughtworks.com/cruise11/cruise-server-1.1-5477.noarch.rpm
wget http://download01.thoughtworks.com/cruise11/cruise-agent-1.1-5477.noarch.rpm

install rpm files

 


rpm -i cruise-server-1.1-5477.noarch.rpm 
using default settings from /etc/default/cruise-server
Started Cruise Server on http://domU-XX-XX-XX-XX-XX-XX:8153/cruise[  OK  ]

add your key which was sent by mail:

in the pipelines panel add your git repo and check connection.

first you might get:

 


Repository git@github.com:user/my-repo.git not found! : 
Error performing command: --- Command --- 
git ls-remote git@github.com:user/repo.git --- Environment --- 
{} --- INPUT ---- --OUTPUT --- --- ERROR --- ERROR: Host key verification failed. 
ERROR: fatal: The remote end hung up unexpectedly ---

ssh into your centos amazon instance and do exactly that: accept the host-key:

 


git ls-remote git@github.com:user/my-repo.git 
The authenticity of host 'github.com (65.74.177.129)' can't be established.
RSA key fingerprint is 16:27:ac:a5:76:28:2d:36:63:1b:56:4d:eb:df:a6:48.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,65.74.177.129' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: The remote end hung up unexpectedly

check the connection again. the next error you probably will get is:

 


Repository git@github.com:user/my-repo.git not found! : 
Error performing command: --- Command --- git ls-remote git@github.com:user/repo.git 
--- Environment --- {} --- INPUT ---- --OUTPUT --- 
--- ERROR --- ERROR: Permission denied (publickey). ERROR: fatal: 
The remote end hung up unexpectedly ---

so we need a public/private keypair.
follow the steps in providing-your-ssh-key
at github. No everything should work ok.

define your pipeline and add the cruise agent

 


rpm -i cruise-agent-1.1-5477.noarch.rpm 
Now please edit /etc/default/cruise-agent and set CRUISE_SERVER to the IP address of your Cruise Server.
Once that is done start the Cruise Agent with '/etc/init.d/cruise-agent start'
[root@domU-XX-XX-XX-XX-XX-XX:~/cruise] /etc/init.d/cruise-agent start
using default settings from /etc/default/cruise-agent
Started Cruise Agent.                                      [  OK  ]

Technorati Tags: ami, aws, cruise, cruise control, ec2, environment 2.0, git, github

Tomcat character enconding UTF-8

Thu, 09 Oct 2008 14:45:00 GMT

As with EVERY project sooner or later I encounter character encoding problems. This time everything seemed to be ok. UTF-8 settings everywhere I can think of. Did I miss something?

On the development environment it worked well, but on production UTF-8 pages showed off with this annoying ?? character sequences instead of german umlauts. What did I miss?

Java seems do rely on default character enconding settings itself. Passing -Dfile.encoding=UTF8 @ with @JAVA_OPTS did the trick :-)

Technorati Tags: java, tomcat

Howto fork a repo on github and submit patches to the author

Tue, 07 Oct 2008 22:21:00 GMT

It’s really easy. Some Documentation can be found at

http://github.com/guides/fork-a-project-and-submit-your-modifications
http://github.com/guides/changing-your-origin
http://railsontherun.com/2008/3/3/how-to-use-github-and-submit-a-patch

click on the fork button on github
clone it from there
add the old origin as e.g. upstream
make the changes
push to own repo
klick on the pull request button.

now that is community developing. great stuff

Technorati Tags: git, github

How to use highline menu in Capistrano

Sat, 20 Sep 2008 23:48:00 GMT

Sometimes invoking a task in Capistrano requires to input data. A very comfortable approach is to use highline since it is build for this purpose.


    task :ask_for_environment do
      set :environments, [:production,:testing,:development]
      set(:environment, Capistrano::CLI.ui.choose do |menu|

          environments.each do |choice|
            menu.choice choice do set(:environment,choice)
            end
          end
        end
       
      ) unless exists?(:environment)
      
      unless environments.include?(environment.to_sym)  
        puts "".rjust(70,'-') << "\nERROR: environment \"#{environment}\" is NOT a valid option\n" << "".rjust(70,'-')
        exit 1
      end

      puts "#{environment}"      
    end

I think this code is self explanatory (more or less ;-). Capistrano::CLI.ui. returns a highline object we can work with. For more examples on how to use highline visit the highline svn repo This can then be combined with any task you like through the before hook


before "deploy", "ask_for_environment"

The unless exists?(:environment) lets one bypass the menu through setting the parameter at the invocation line ( -s environment=production)

Technorati Tags: capistrano

Puppet and external node definition through icpuppet from iClassify

Sat, 13 Sep 2008 22:55:00 GMT

Puppet is great for “implementing” server infrastructure. Puppet has a concept of nodes and classes.


node 'webserver.testing.com' { 
        include apache 
} 

class apache { 
 
$packagelist = ["httpd", "webalizer", "mod_ssl"] 
 
package { $packagelist: 
       ensure => "installed" 
}
...
}

This example shows how the hostname webserver.testing.com is bound to the class apache. This approach is a kind of static. To get to a more dynamic nature of implementing things we can use icpuppet which is bundled with iClassify. To enable it in puppet include


    external_nodes = "/path/to/icpuppet"
    node_terminus = exec
</pre>

in you /etc/puppet/puppet.conf. I modified my icpuppet to include my iClassify credentials directly:


config = {
  :server    => 'http://localhost:5000',
  :user      => 'admin',
  :passwd    => 'admin'
}
</pre>

I also modified the search string to fit my needs


 results = client.search("hostname:\"#{hostname}\" OR fqdn:\"#{hostname}\" OR puppet_host_id:\"#{hostname}\"")
</pre>

Here you can implement ANYTHING which gives back the node you are searching for.

To test it, just invoke the icpuppet binary with a potential hostname. Already registered hostnames can be retrieved from puppet with


puppetca --list --all
domu-12-XX-38-XX-42-XX.compute-1.internal

+ domu-XX-XX-XX-00-XX-c6.compute-1.internal



an invocation of icpuppet yields:


icpuppet domu-XX-XX-38-00-8d-XX.compute-1.internal
--- 
parameters: 
  processorcount: "1"
  kernel: Linux
  puppet_host_id: domu-XX-XX-38-00-8d-XX.compute-1.internal
  .....
classes: 

- base

- apache



This result gets interpreted by puppet as if we would define a node domu-XX-XX-38-00-8d-XX.compute-1.internal including the classes base and apache


node 'domu-XX-XX-38-00-8d-XX.compute-1.internal' { 
        include base
        include apache 
} 
</pre>

Additionally all parameters can be accessed directly in puppet as normal variables. E.g. puppet_host_id can be accessed with $puppet_host_id. Another great feature is that all nodes are “tagged” with their classes. which enables for statements like:


 if tagged(apache){
                notice("This is a webserver")
}

Technorati Tags: environment 2.0, iclassify, puppet

Bootstraping ec2 with Puppet, iClassify, and Capistrano

Sat, 06 Sep 2008 20:05:00 GMT

Searching through github ( http://github.com/search?q=puppet) I came across a very similar approach on howto bootstrap EC2 images with capistrano puppet and iclassify. ec2-cap is build on top of the Rightscale Amazon EC2 gem (http://rightaws.rubyforge.org/)

http://github.com/adamhjk/ec2-cap/tree/master

The main idea behind this is to

start an EC2 instance (based on a simple declarative description) with capistrano
trigger a register process from this instance to the puppet master.
sign the certificate on the puppet master
pull the default configuration on the newly created host from the puppet master
register the new host with the iClassify server
tag the new host for the desired purpose (e.g. app for application server)
pull the node specific configuration from the puppet master

Technorati Tags: aws, capistrano, cloud, ec2, iclassify, puppet

Automated Environment: Puppet, iClassify, and Capistrano

Tue, 26 Aug 2008 22:03:00 GMT

There is a new hype: cloud computing. Your web site traffic grows? Add a new application server. Want to increase availability? Add a new web server. Among others Amazon offers a whole bunch of infrastructure to support these needs. But it is not enough to fire up a server. The web server should include apache, the app server should include tomcat. The deployment should be aware of the new app server. This is where Puppet, iClassify, and Capistrano come into play. Puppet does the configuration management, installs your desired web and app server.

Depending on the "server class" it automagically can install packages, configurations, etc. iClassify plugs into this nicely: each new server registers with iClassify, providing the vital information of itself (yes, its scriptable and there already is script for exposing all relevant EC2 data of an instance). Once tagged with a "server class" Puppet can do its work. iClassify nicely integrates with Capistrano, so deployment tasks can be rolled out through the result of searches in the iClassify database. "Deploy to all HOSTS in $CLOUD where hasTomcat=TRUE".

Following is a writeup of the Puppet and iClassify installation on a Rightscale CentOS 5.0 Amazon EC2 ami:

It is mainly based on https://wiki.hjksolutions.com/display/IC/Install+Instructions, so credits go there :-)

Install iClassify

http://reductivelabs.com/trac/puppet/wiki/InstallationGuide
http://ifireball.wordpress.com/docs/howto-install-puppet-on-centos-50/

cd /etc/yum.repos.d wget http://people.redhat.com/dlutter/yum/dlutter.repo

Enable the proper repository, edit the dlutter.repo file, find the [dlutter-rhel5] section and set enabled=1

vi dlutter.repo

yum install puppet-server yum install ruby-rdoc

Create a minimal manifest file in /etc/puppet/manifests with the following content:

# The name of the system running the file server

$server = "pmaster"

node default {

}

vi /etc/puppet/manifests/minimal /etc/init.d/puppetmaster start chkconfig puppetmaster on

Install iClassify

http://github.com/adamhjk/iclassify/tree/master/INSTALL
https://wiki.hjksolutions.com/display/IC/Install+Instructions

install rails and rake (already on ami)
review this (capistrano tasks for puppet/iclassify):

http://reductivelabs.com/trac/puppet/wiki/RegenerateSSL

recipes for puppet (should try out the iclassify thing):
http://github.com/apenney/puppet/tree/master
contact: If you have any specific questions there's an irc channel on freenode (#puppet), and the major developers are all there and it's pretty easy to get advice (if it's in the work week between 9-5est I'll always help if I can.)

presentations:
http://reductivelabs.com/trac/puppet/wiki/PuppetPresentations

other blog entries
http://plathrop.tertiusfamily.net/blog/category/tools/puppet/
http://www.red91.com/2008/07/09/puppet-setup-test-notes

install iClassify

gem list|grep rails gem list|grep rake

install builder, UUID Tools. Mongrel, Highline

gem install builder uuidtools mongrel highline ruby-net-ldap yum install mysql mysql-devel

/etc/init.d/mysqld start /usr/bin/mysqladmin -u root password 'mysql' mysqladmin create iclassify_production -p mysql iclassify_production -p GRANT ALL ON iclassify_production.* TO 'iclassify'@'localhost' IDENTIFIED BY 'yourpass';

http://smangal.wordpress.com/2007/05/17/mysql-gem-on-centos/
gem install mysql -- --with-mysql-include=/usr/include/mysql --with-mysql-lib=/usr/lib/mysql

wget https://wiki.hjksolutions.com/download/attachments/295039/mongrel_runit-0.2.0.gem gem install mongrel_runit-0.2.0.gem

yum install git yum install facter

http://rubyworks.rubyforge.org/installation/centos.html
wget http://rubyworks.rubyforge.org/RubyWorks.repo

RubyWorks.repo /etc/yum.repos.d/
change baseurl to (does read to 5.0 --> no web site)

baseurl=http://rubyworks.rubyforge.org/redhat/5/RPMS/$basearch
in
vi /etc/yum.repos.d/RubyWorks.repo yum install runit

install IClassify

choose your ICLASSIFY_HOME directory
git clone git://git.hjksolutions.com/iclassify iclassify cd iclassify/

rake iclassify:install ICBASE=/srv/iclassify ICUSER=apache ICGROUP=apache DBUSER=iclassify DBPASS=yourpass chown -R apache:apache /srv/iclassify/*

mkdir /etc/mongrel_runit cp /srv/iclassify/examples/mongrel_runit_iclassify.yml /etc/mongrel_runit/iclassify.yml

change user and group to apache in
/etc/mongrel_runit/iclassify.yml

mongrel_runit -c /etc/mongrel_runit/iclassify.yml create chmod a+x /etc/init.d/mongrel_runit ln -s /etc/init.d/mongrel_runit /etc/init.d/iclassify /etc/init.d/iclassify -v status

mkdir -p /etc/sv/iclassify-solr/log/main cp /srv/iclassify/examples/solr-run /etc/sv/iclassify-solr/run cp /srv/iclassify/examples/solr-log /etc/sv/iclassify-solr/log/run chmod a+x /etc/sv/iclassify-solr/run /etc/sv/iclassify-solr/log/run ln -s /etc/sv/iclassify-solr /var/service

sv binary is in sbin (in this centos install)
ln -s /usr/sbin/sv /etc/init.d/iclassify-solr

change JAVA_HOME to/usr/java/default vi /etc/sv/iclassify-solr/run

solr logfiles are in
/var/service/iclassify-solr/log/main/current

for apache:

add a /etc/httpd/conf.d/iClassify.conf

<VirtualHost *:*> ProxyPass / http://localhost:5000/ ProxyPassReverse / http://localhost:5000/ </VirtualHost>

in your ICLASSIFY_HOME

./bin/icagent -d ./icangent -s http://localhost:5000 -n

to show the status

and if it fit your needs:

./bin/icagent -d ./icangent -s http://localhost:5000

now you are ready. log into the iclassify server and enjoy :-)

install the ssl for iClassify:

http://www.electrictoolbox.com/mod-ssl-apache-centos-5/

Technorati Tags: amazon, aws, capistrano, cloud, ec2, environment 2.0, high availability, iclassify, puppet

Apache Webserver and Tomcat Virtual Hosts

Mon, 25 Aug 2008 17:02:00 GMT

Problem description: You want to server multiple web applications with one Tomcat instance. The brute force variant is to just "rewrite" or ProxyPass the request from e.g. www.myhost.com to the local tomcat with localhost:8080/myhost. But this leads to context urls of ressources in the served html pages. You can either deploy you webapp in the ROOT context (but there is just ONE ROOT context, so this does not work). So let us use virtual hosts on both, the apache webserver and the tomcat servlet container.

create a app location in $CATALINA_HOME
mkdir www.your-server-name.net

deploy the webapp in ROOT context

edit server.xml according
http://confluence.atlassian.com/display/DOC/Guide+to+using+Apache+Tomcat%27s+Virtual+Hosts
http://tomcat.apache.org/tomcat-6.0-doc/

in Tomcat server.xml

edit Apache config according:

In Apache conf
http://confluence.atlassian.com/display/DOC/Using+Apache+with+virtual+hosts+and+mod_proxy

<VirtualHost *>
ServerName www.your-server-name.net
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://www.internal.your-server-name.net:8080/
ProxyPassReverse / http://www.internal.your-server-name.net:8080/
<Location />
Order allow,deny
Allow from all
</Location>
</VirtualHost>

and in /etc/hosts

127.0.0.1 localhost www.internal.your-server-name.net

restart tomcat
restart apache

that's it :-)

Technorati Tags: apache, cloud, tomcat

Amazon Elastic Block Store

Fri, 22 Aug 2008 20:41:00 GMT

There is a new kid on the block: Amazon Elastic Block Store

The new service fills one of the last missing links in the Amazon Webservices. It provides for persistent, high-performance, high-availability block-level storage which you can attach to a running instance of EC2.http://aws.typepad.com/aws/2008/08/amazon-elastic.html. And there is a great tutorial by Eric Hammond on how to use this new service with Mysql Running MySQL on Amazon EC2 with Elastic Block Store

Technorati Tags: amazon, aws, cloud, ebs, ec2

Poolparty, a promising tool for cloud computing

Thu, 21 Aug 2008 18:13:00 GMT

Poolparty is a very interesting project. It promises easy cloud computing with Amazon EC2. The project undergoes a heavy rewrite of its core, so we ought to stay tuned :-)

Ressource links are as follows:

The homepage:
http://poolpartyrb.com/

The repository can be found at:
http://github.com/auser/poolparty/tree/master

Technorati Tags: amazon, aws, cloud, ec2, environment 2.0

High-Availability Load Balancer (With Failover and Session Support) With HAProxy

Wed, 20 Aug 2008 16:43:00 GMT

HAProxy seems very promising as a production use software loadbalancer. The only drawback with this solution is that one still has to use some extra piece of middleware to terminate a ssl session (stunnel, apache).

Setting Up A High-Availability Load Balancer (With Failover and Session Support) With HAProxy/Keepalived On Debian Etch
http://www.howtoforge.com/haproxy_loadbalancer_debian_etch

Using HA Proxy with Amazon EC2
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1639&categoryID=100

haproxy documentation
http://haproxy.1wt.eu/download/1.2/doc/haproxy-en.txt

Technorati Tags: amazon, aws, cloud, ec2, high availability