Author: Tamer Embaby

Install frequently used packages in fresh Ubuntu using Ansible

There is always need for installing frequently used packages in a freshly installed Ubuntu Linux. I personally use this when I get new laptop, but it can be changed to manage server or virtual machine instances. My main goal is to play with ansible and use it in something practical for me.

  • Install ansible
$ sudo apt-get install ansible ansible-doc
  • Add localhost to ansible hosts file

The file /etc/ansible/hosts should have the following added:

[local]
127.0.0.1
  • Ping the server

Ping the server to make sure configuration is good. SSH will ask to localhost to known_hosts file.

$ ansible all -m ping
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ECDSA key fingerprint is d2:79:49:f4:c4:8d:13:83:50:ce:6c:94:5a:1b:d3:31.
Are you sure you want to continue connecting (yes/no)? yes
127.0.0.1 | success >> {
"changed": false,
"ping": "pong"
}
  • Write the basic-packages.yml file

Write YAML file containing ansible command to install the packages, the file name is basic-packages.yml and contains the following*:

---
 
- name: install packages on a fresh Ubuntu
  hosts: local
  user: "{{ username }}"
  sudo: yes
 
  tasks:
    - name: install all packages
    apt: name={{ item }} state=present
    with_items:
    - apache2
    - nginx-full
    - gnupg
    - ansible-doc
    - apache2
    - audacity
    - avidemux
    - awscli
    - checkbox-gui
    - cheese
    - compizconfig-settings-manager
    - compiz-plugins-extra
    - curl
    - cvs
    - dos2unix
    - ec2-api-tools
    - enigmail
    - exfat-fuse
    - exfat-utils
    - ffmpeg
    - filezilla
    - fonts-arabeyes
    - fonts-droid
    - gcc
    - geoip-bin
    - gimp
    - google-chrome-stable
    - gpac
    - hunspell-ar
    - hunspell-en-ca
    - language-pack-ar-base
    - language-pack-gnome-ar-base
    - libavcodec-extra
    - libjs-jquery
    - libnss-myhostname
    - mysql-server
    - nautilus-dropbox
    - nautilus-image-converter
    - nmap
    - openjdk-7-jdk
    - openssh-server
    - php5-cli
    - php5-mysql
    - python-apport
    - python-lockfile
    - python-requests
    - rdesktop
    - seahorse-nautilus
    - subversion
    - thunderbird-locale-ar
    - thunderbird-locale-en-gb
    - traceroute
    - ttf-mscorefonts-installer
    - ubuntu-restricted-extras
    - ubuntu-session
    - unity-control-center
    - unity-tweak-tool
    - unrar
    - vim
    - virtualbox-4.3
    - vlc
    - vpnc
    - whois

So ansible will attempt to install the above packages on the fresh Ubuntu Desktop.

* Get list of manually installed package explained here

  • Dry run ansible playbook

Run the ansible playbook to make sure the configuration doesn’t have any syntax issues and ansible can run the module correctly:

$ ansible-playbook --check basic-packages.yml -e username=myusername --ask-sudo-pass
sudo password:
 
PLAY [install packages on a fresh Ubuntu] *************************************
 
GATHERING FACTS ***************************************************************
ok: [127.0.0.1]
 
TASK: [install all packages] **************************************************
changed: [127.0.0.1] => (item=apache2,nginx-full,gnupg,ansible-doc,apache2,audacity,avidemux,awscli,checkbox-gui,cheese,compizconfig-settings-manager,compiz-plugins-extra,curl,cvs,dos2unix,ec2-api-tools,enigmail,exfat-fuse,exfat-utils,ffmpeg,filezilla,fonts-arabeyes,fonts-droid,gcc,geoip-bin,gimp,google-chrome-stable,gpac,hunspell-ar,hunspell-en-ca,language-pack-ar-base,language-pack-gnome-ar-base,libavcodec-extra,libjs-jquery,libnss-myhostname,mysql-client,mysql-server,nautilus-dropbox,nautilus-image-converter,nmap,openjdk-7-jdk,openssh-server,php5-cli,php5-mysql,python-apport,python-lockfile,python-requests,rdesktop,seahorse-nautilus,subversion,thunderbird-locale-ar,thunderbird-locale-en-gb,traceroute,ttf-mscorefonts-installer,ubuntu-restricted-extras,ubuntu-session,unity-control-center,unity-tweak-tool,unrar,vim,virtualbox-4.3,vlc,vpnc,whois)
 
PLAY RECAP ********************************************************************
127.0.0.1 : ok=2 changed=1 unreachable=0 failed=0
 
$

I use –ask-sudo-pass to pass sudo user password to ansible on remote (local in this case).

  • How to use it

On a new installed Ubuntu, install ansible, cope the playbook file to the machine, and run the playbook.

This can be extended of course to changing configuration files using ansible Jinja2 templates.

List manually installed package in Ubuntu

Here is a one-liner to list all packages manually installed since the fresh installation of my Ubuntu laptop:

$ comm -23 <(apt-mark showmanual | sort -u) <(gzip -dc /var/log/installer/initial-status.gz | sed -n 's/^Package: //p' | sort -u)
ansible
ansible-doc
apache2
audacity
avidemux
...
vim
virtualbox-4.3
vlc
vpnc
whois
$

Low letancy/High performance RHEL

Recently I worked with a customer who requires high performance servers and near real time response for their business application. They were very concerned about the power saving options in the CPU and other server components and were advised by their application vendor to disable these feature.

The customer is running Red Hat Enterprise Linux 6 on a 28 x (128 GB RAM, 2 x 8-core Intel Xeon CPU, SSD harddisks) – quite huge specs!

Since the application vendor have milliseconds-based SLA with the customer, they urged the customer to use the following values to disable power saving at CPU level:

intel_idle.max_cstate=0
processor.max_cstate=0
  • Background

CPU’s use a transition multiple levels of power-saving states (called C-states), for example it changed from full functioning state (called C0) to a lesser state where some CPU components are shut down, if it remains idle it will transit to deeper C-State and shut down more components. Until CPU is assigned a new task/application it starts to wake up into higher C-States it fully functioning again.

Unfortunately, transitioning from deeper power saving states back to a fully powered up state takes time and can introduce undesired application delays when powering on components.

  • How to really do it in RHEL?

The correct way to do this is to pass the processor.max_cstate=1 and idle=poll parameters to the kernel on server startup, this will disable CPU power saving, lower latency and increase performance. The impact to this is much more power consumed, much heat generated, and the server has to be monitored to make sure it’s OK, and has to be cooled down all the time, hence even more power consumption.

However, Red Hat provides the interface /dev/cpu_dma_latency which control processor C-State and can prevent CPU from transitioning to deeper power saving modes. Simply any process can open the file /dev/cpu_dma_latency and write 0 to it to accomplish the same effect (preventing CPU from entering deeper C-State), and this effect remains as long as the process keeps the file open.

So /dev/cpu_dma_latency can control the above settings in much better way, this interface is controlled with the package tuned, and controlled with the command tuned-adm. The idea about it is it defines power saving profiles that you can change based on your needs, so for example to disable CPU power saving to decrease latency and increase performance we would use:

# tuned-adm profile latency-performance

Which will do the exact effect required by application vendor, as well as preventing other power saving parameters in the system like HD.

Returning back to normal powersaving state can be achieved with:

# tuned-adm profile default

or even

# Shutdown tuned
# tuned-adm off

So, to still save power and help the server be in reasonable temperature and not to emit too much heat, enabling latency-performance profile before business hours and return it back to normal server behavior after business hours (via crontab) would be much better solution in my opinion since it will give the desired application performance and yet conserve power in non business time and avoid high power consumption.

Tuned can be installed from Red Hat repository or from RHEL CD.