Saturday, June 16, 2012

自动化部署工具Chef实战

install

install

1 说明

Chef是一个基础设施管理工具,用于集中和简化配置的管理,已经可以用于复杂的企业环境中,跑在多种平台之上

  • 本文档只是将其它地方的一些资料做了整理,以便新手可以快速搭建起chef
  • 使用三台虚拟机分别做为server,workstation,client
  • 内容主要包括以下:
    • 搭建一个Server模式的Chef环境
    • 创建一个cookbook
    • 编写一个简单的recipe配置client的ntp服务
  • 本文档只适用于Ubuntu 12.04

2 准备工作

  • 修改主机名
    Chef需要主机设置一个完整的主机名(FQDN) 修改完后用hostname -f确认
    hostname -f
    server.chefdemo.com
    
  • 添加Opscode仓库
    echo "deb http://apt.opscode.com/ `lsb_release -cs`-0.10 main" | sudo tee /etc/apt/sources.list.d/opscode.list
    
  • 添加GPG密钥
    sudo mkdir -p /etc/apt/trusted.gpg.d
    gpg --keyserver keys.gnupg.net --recv-keys 83EF826A
    gpg --export packages@opscode.com | sudo tee /etc/apt/trusted.gpg.d/opscode-keyring.gpg > /dev/null
    
  • 更新源
    sudo apt-get update
    
  • 安装opscode-keyring
    sudo apt-get install opscode-keyring # permanent upgradeable keyring
    
  • 升级已经安装的软件包
    sudo apt-get upgrade
    

3 安装及配置Chef Server

  • 安装配置chef-server
    sudo apt-get install chef chef-server
    
    要安装300多个包,需要一些时间
    安装过程会提示你进行配置
    首先是配置chef-client(前面的安装方法会安装和启动chef-client,如果不需要这个服务可以关闭它)
    ┌───────────────────────┤ Configuring chef ├───────────────────────┐
    │  This is the full URI that clients will use to connect to the    │  
    │  server.                                                         │ 
    │  .                                                               │  
    │  This will be used in /etc/chef/client.rb as 'chef_server_url'.  │  
    │                                                                  │     
    │ URL of Chef Server (e.g., http://chef.example.com:4000):         │           
    │                                                                  │                     
    │ http://server.chefdemo.com:4000_________________________________ │                                           
    │                                                                  │                                                                              
    │                              <Ok>                                │          
    │                                                                  │
    └──────────────────────────────────────────────────────────────────┘  
    
    由于chef-server在同一台主机上,可以输入localhost
    接下来会提示输入RabbitMQ服务的密码(chef默认使用chef用户和/chef vhost)
    ┌────────────────────────────┤ Configuring chef-solr ├─────────────────────────────┐                                           
    │  Set the password for the chef user in the AMQP server queue. Use                │                                           
    │  RabbitMQ's rabbitmqctl program to set this password. The default user           │                                           
    │  and vhost are assumed (chef and /chef, respectively).                           │                                           
    │  .                                                                               │                                           
    │  RabbitMQ does not have the capability to read the password from a file, and     │                                           
    │  this will be passed via "" on the command-line. As such, do not use shell       │                                           
    │  meta-characters that could cause errors such as !.                              │                                           
    │  .                                                                               │                                           
    │  This will be used in /etc/chef/solr.rb and /etc/chef/server.rb as 'amqp_pass'.  │                                           
    │                                                                                  │                                           
    │ New password for the 'chef' AMQP user in the RabbitMQ vhost "/chef":             │                                           
    │                                                                                  │                                           
    │ ******__________________________________________________________________________ │                                           
    │                                                                                  │                                           
    │                                      <Ok>                                        │                                           
    │                                                                                  │                                           
    └──────────────────────────────────────────────────────────────────────────────────┘ 
    
    接着会提示输入Chef Server WebUI admin用户的密码(用于登陆Chef图形管理界面)
    ┌───────────────────────┤ Configuring chef-server-webui ├───────────────────────┐                                            
    │  This sets a temporary first-use password to log into the Chef Server WebUI   │                                            
    │  as the 'admin' user for the first time. Once logged in, the password should  │                                            
    │  be changed immediately.                                                      │                                            
    │  .                                                                            │                                            
    │  Once the chef-server-webui process is running, login using the username      │                                            
    |  'admin' using the password set here.                                         │                                            
    │  .                                                                            │                                            
    │  If a password is not entered, the webui default password for 'admin' will    │                                            
    │  be used, which is displayed on the webui home page. The password must be     │                                            
    │  at least 6 characters or the webui will not start properly.                  │                                            
    │  .                                                                            │                                            
    │  This will be used in /etc/chef/webui.rb as 'web_ui_admin_default_password'.  │                                            
    │  .                                                                            │                                            
    │                                                                               │                                            
    │ New password for the 'admin' user in the Chef Server WebUI:                   │                                            
    │                                                                               │                                            
    │ ******_______________________________________________________________________ │                                            
    │                                                                               │                                            
    │                                    <Ok>                                       │                                            
    │                                                                               │                                            
    └───────────────────────────────────────────────────────────────────────────────┘
    
    到此chef-server已经安装配置完成,现在登陆http://server.chefdemo.com:4040 应该可以看到WebUI了
  • 验证安装
    下表出自: Installing Chef Server Manually
    |-------------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    | Name              | Listen Port | Example Program Name in ps (Erlang programs truncated)                                                                                                                                             |
    |-------------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    | Chef Server       |        4000 | merb : chef-server (api) : worker (port 4000)                                                                                                                                                      |
    | Chef Server WebUI |        4040 | merb : chef-server-webui : worker (port 4040)                                                                                                                                                      |
    | CouchDB           |        5984 | beam.smp -Bd -K true – -root /usr/local/lib/erlang -progname erl – -noshell -noinput -couch_ini /usr/local/etc/couchdb/default.ini /usr/local/etc/couchdb/local.ini -s couch                       |
    | RabbitMQ          |        5672 | {{beam.smp -W w -K true -A30 – -root /usr/local/lib/erlang -progname erl – -noshell -noinput -s rabbit -sname rabbit -rabbit tcp_listeners [{"0.0.0.0", 5672}]}}                                   |
    | Chef Solr         |        8983 | /usr/bin/java -Xmx250M -Xms250M -Dsolr.data.dir=/opscode/chef/features/data/solr/data -Dsolr.solr.home=/opscode/chef/features/data/solr/home -jar /opscode/chef/features/data/solr/jetty/start.jar |
    | Chef Expander     |        none | ruby ./chef-solr/bin/chef-expander -c /etc/chef/solr.rb -l debug                                                                                                                                   |
    |-------------------+-------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
    

4 安装及配置Chef Workstation

  • 安装Chef,Ruby和依赖包
    sudo apt-get install chef ruby ruby-dev libruby ri build-essential wget ssl-cert curl
    
    安装过程中会提示你配置chef
    ┌───────────────────────┤ Configuring chef ├───────────────────────┐                                                   
    │  This is the full URI that clients will use to connect to the    │                                                   
    │  server.                                                         │                                                   
    │  .                                                               │                                                   
    │  This will be used in /etc/chef/client.rb as 'chef_server_url'.  │                                                   
    │                                                                  │                                                   
    │ URL of Chef Server (e.g., http://chef.example.com:4000):         │                                                   
    │                                                                  │                                                   
    │ http://server.chefdemo.com:4000_________________________________ │                                                   
    │                                                                  │                                                   
    │                              <Ok>                                │                                                   
    │                                                                  │                                                   
    └──────────────────────────────────────────────────────────────────┘ 
    
    workstation一般不需要chef-client,只是安装chef的时候它会自动安装,可以手动关掉它
    sudo update-rc.d chef-client disable
    /etc/init.d/chef-client stop
    

5 安装配置Chef Client

  • 安装配置chef-client
    sudo apt-get install chef
    
    安装过程会提示输入会提示输入chef-server的url,和前面一样输入即可
    安装完成后chef-client已经启动了,但查看日志你会发现有报错:
    root@client1:~# tail /var/log/chef/client.log 
    # Logfile created on Thu Jun 14 13:19:54 +0800 2012 by logger.rb/1.2.6
    [Thu, 14 Jun 2012 13:19:54 +0800] INFO: Daemonizing..
    [Thu, 14 Jun 2012 13:19:54 +0800] INFO: Forked, in 7380. Privileges: 0 0
    [Thu, 14 Jun 2012 13:20:11 +0800] INFO: *** Chef 0.10.10 ***
    [Thu, 14 Jun 2012 13:20:12 +0800] INFO: Client key /etc/chef/client.pem is not present - registering
    [Thu, 14 Jun 2012 13:20:12 +0800] WARN: Failed to read the private key /etc/chef/validation.pem: #<Errno::ENOENT: No such file or directory - /etc/chef/validation.pem>
    [Thu, 14 Jun 2012 13:20:12 +0800] ERROR: Chef::Exceptions::PrivateKeyMissing: I cannot read /etc/chef/validation.pem, which you told me to use to sign requests!
    [Thu, 14 Jun 2012 13:20:12 +0800] FATAL: Stacktrace dumped to /var/cache/chef/chef-stacktrace.out
    [Thu, 14 Jun 2012 13:20:12 +0800] ERROR: Sleeping for 1800 seconds before trying again
    
    chef-client的chef-server之间的通信是通过密钥的
    当chef-client启动的时候,它首先会检查/etc/chef/client.pem文件是否存在,如果存在,就用这个私钥去连chef-server
    如果不存在,它就会借用/etc/chef/validation.pem连到chef-server进行注册
    我们只需要从chef-server上复制validation.pem过来即可
    # 在client1上
    scp root@server.chefdemo.com:/etc/chef/validation.pem /etc/chef/
    chmod +r /etc/chef/validation.pem
    
    然后启动服务,应该可以看到注册成功了
    [Thu, 14 Jun 2012 13:34:19 +0800] INFO: Client key /etc/chef/client.pem is not present - registering
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: HTTP Request Returned 404 Not Found: Cannot load node client1.chefdemo.com
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Run List is []
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Run List expands to []
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Starting Chef Run for client1.chefdemo.com
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Running start handlers
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Start handlers complete.
    [Thu, 14 Jun 2012 13:34:21 +0800] INFO: Loading cookbooks []
    [Thu, 14 Jun 2012 13:34:21 +0800] WARN: Node client1.chefdemo.com has an empty run list.
    [Thu, 14 Jun 2012 13:34:22 +0800] INFO: Chef Run complete in 0.910796 seconds
    [Thu, 14 Jun 2012 13:34:22 +0800] INFO: Running report handlers
    [Thu, 14 Jun 2012 13:34:22 +0800] INFO: Report handlers complete
    
    这时/etc/chef/client.pem也应该自动生成了

6 配置knife命令工具

knife是管理chef的命令行工具,一般管理员都是在workstation上用knife对chef进行管理
  • 首先在chef-server上配置命令行工具
    # run on chef-server
    mkdir -p ~/.chef
    sudo cp /etc/chef/validation.pem /etc/chef/webui.pem ~/.chef
    sudo chown -R $USER ~/.chef
    knife configure -i
    
    WARNING: No knife configuration file found
    Where should I put the config file? [/root/.chef/knife.rb] 
    Please enter the chef server URL: [http://server.chefdemo.com:4000] 
    Please enter a clientname for the new client: [root] 
    Please enter the existing admin clientname: [chef-webui] 
    Please enter the location of the existing admin client's private key: [/etc/chef/webui.pem] .chef/webui.pem
    Please enter the validation clientname: [chef-validator] 
    Please enter the location of the validation key: [/etc/chef/validation.pem] .chef/validation.pem
    Please enter the path to a chef repository (or leave blank): 
    Creating initial API user...
    Created client[root]
    Configuration file written to /root/.chef/knife.rb
    
  • 验证配置
    # run on chef-server node
    knife client list
      chef-validator
      chef-webui
      chefadmin
      client1.chefdemo.com
      root
      server.chefdemo.com
    
  • 在chef-server上创建一个client帐户(此例中为chefadmin)
    echo "export EDITOR=vim" >> ~/.bashrc
    source ~/.bashrc
    knife client create chefadmin -n -a -f /tmp/chefadmin.pem
    
    上条命令会用$EDITOR打字一个文件,保存即可
    可以查看刚刚创建的client
    # stil run on chef-server node
    knife client show chefadmin
    _rev:        2-895e0cdaa5d15109fa0a5a933b55b063
    admin:       true
    chef_type:   client
    json_class:  Chef::ApiClient
    name:        chefadmin
    public_key:  -----BEGIN RSA PUBLIC KEY-----
                 MIIBCgKCAQEAt0BlxDudQqys7pB1FoXNPAIKd5Lik0TkWIxaSQ5m69wBWgkQNHwX
                 GVCTn7GMLnSMmjqNR59TGhMrEXjjCFlDYAFxpVYyaD1oR3K7pNx5vyUkUqWDyv37
                 q5EeLW1z18pfm/xxMxL/eRCpSkWoJrP77/2Z8DDqyZlh7b9ZBJ7UUG7OskU3qMse
                 ngUGL6rRRYjSg8ELBLXJhpetbejvMOnUafGcvppnIj79UrA0fcMgrTpYWxb47Xo6
                 fMyKQ0DjDXuUvgj7mrnIOox6NhRnamVJYbFyiLPIheaonEOBVr7TeASprqF3nIqm
                 V/XyNmAGwtmYE2Qne4UorwJFRORU9zz0OwIDAQAB
                 -----END RSA PUBLIC KEY-----
    
  • 将生成的key文件复制到workstation节点上
    # now run on workstation node
    mkdir ~/.chef
    scp server.chefdemo.com:/tmp/chefadmin.pem ~/.chef/chefadmin.pem
    
    knife configure
    
    WARNING: No knife configuration file found
    Where should I put the config file? [/root/.chef/knife.rb] 
    Please enter the chef server URL: [http://workstation.chefdemo.com:4000] http://server.chefdemo.com:4000
    Please enter an existing username or clientname for the API: [root] chefadmin
    Please enter the validation clientname: [chef-validator] 
    Please enter the location of the validation key: [/etc/chef/validation.pem] 
    Please enter the path to a chef repository (or leave blank): 
    *****
    
    You must place your client key in:
      /root/.chef/chefadmin.pem
    Before running commands with Knife!
    
    *****
    
    You must place your validation key in:
      /etc/chef/validation.pem
    Before generating instance data with Knife!
    
    *****
    Configuration file written to /root/.chef/knife.rb
    
    验证:
    knife client list
    

7 Chef示例(ntp)

  • 创建一个ntp cookbook
    # run on workstation or other host with knife configured
    knife cookbook create ntp
    
    可以在后面加-o指定路径,默认保存在/var/chef/cookbooks/下
    tree /var/chef/cookbooks/ntp
    /var/chef/cookbooks/ntp
    ├── attributes
    ├── definitions
    ├── files
    │   └── default
    ├── libraries
    ├── metadata.rb
    ├── providers
    ├── README.md
    ├── recipes
    │   └── default.rb
    ├── resources
    └── templates
        └── default
    
    10 directories, 3 files
    
  • 创建recipe
    注: 每个cookbook都会有一个名为default的特殊的recipe,它每次都会被执行,适合写一些通用的配置到里面
    我们创建一个新的recipe
    vim /var/chef/cookbooks/ntp/recipes/ntp.rb
    
    package "ntp" do
        action [:install]
    end
    
    template "/etc/ntp.conf" do
        source "ntp.conf.erb"
        variables( :ntp_server => "time.nist.gov" )
        notifies :restart, "service[ntp]"
    end
    
    service "ntp" do
        action [:enable,:start]
    end
    
  • 创建ntp.conf.erb模板文件
    vim /var/chef/cookbooks/ntp/templates/default/ntp.conf.erb
    
    # generated by Chef.
    restrict default kod nomodify notrap nopeer noquery
    restrict -6 default kod nomodify notrap nopeer noquery
    restrict 127.0.0.1
    restrict -6 ::1
    server <%= @ntp_server %>
    server  127.127.1.0     # local clock
    driftfile /var/lib/ntp/drift
    keys /etc/ntp/keys
    
  • 上传cookbook到chef-server
    knife cookbook upload ntp
    
  • 把刚创建的recipe加到client1的run list里
    knife node run list add client1.chefdemo.com 'recipe[ntp::ntp]'
    
  • 在client上重启chef-client并查看结果,如果不成功查看日志进行debug
    # run on client node
    /etc/init.d/chef-client restart
    tail /var/log/chef/client.log
    
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Run List is [recipe[ntp::ntp]]
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Run List expands to [ntp::ntp]
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Starting Chef Run for client1.chefdemo.com
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Running start handlers
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Start handlers complete.
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Loading cookbooks [ntp]
    [Thu, 14 Jun 2012 14:54:46 +0800] INFO: Processing package[ntp] action install (ntp::ntp line 1)
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: package[ntp] installed version 1:4.2.6.p3+dfsg-1ubuntu3.1
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: Processing template[/etc/ntp.conf] action create (ntp::ntp line 5)
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: template[/etc/ntp.conf] backed up to /var/lib/chef/backup/etc/ntp.conf.chef-20120614145458
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: template[/etc/ntp.conf] updated content
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: Processing service[ntp] action enable (ntp::ntp line 11)
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: Processing service[ntp] action start (ntp::ntp line 11)
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: template[/etc/ntp.conf] sending restart action to service[ntp] (delayed)
    [Thu, 14 Jun 2012 14:54:58 +0800] INFO: Processing service[ntp] action restart (ntp::ntp line 11)
    [Thu, 14 Jun 2012 14:54:59 +0800] INFO: service[ntp] restarted
    [Thu, 14 Jun 2012 14:54:59 +0800] INFO: Chef Run complete in 12.839044 seconds
    [Thu, 14 Jun 2012 14:54:59 +0800] INFO: Running report handlers
    [Thu, 14 Jun 2012 14:54:59 +0800] INFO: Report handlers complete
    
Date: 2012-06-21 07:44:43 CST
Author: William
Org version 7.8.02 with Emacs version 23
Validate XHTML 1.0

No comments:

Post a Comment