Welcome!

Microservices Expo Authors: Zakia Bouachraoui, Elizabeth White, Pat Romanski, Liz McMillan, Yeshim Deniz

Related Topics: @DevOpsSummit, Microservices Expo, @CloudExpo, Apache

@DevOpsSummit: Blog Post

XL Deploy and Puppet By @BMoussaud | @DevOpsSummit [#DevOps]

Puppet and XL Deploy can work together if we put each of them in their domain

XL Deploy and Puppet
By Benoit Moussaud

Puppet and XL Deploy can work together if we put each of them in their domain:

  • Puppet manages the provisioning by ensuring the OS and the middleware is correctly configured : This node should have a Tomcat 7.0.42 instance running using a tomcat user and listening en the 8080 port
  • XLD manages the application deployment that takes 2 inputs: a deployment package built by CI tools (Jenkins / TFS) and a environment built by a provisioning tools, e.g. Puppet !

The integration between the two solutions is handled by a module provided by XebiaLabs that will ensure the containers are correctly defined in the XL Deloy repository based on the information managed by Puppet. It uses the REST API offered by the XL Deploy server: so the security permissions are checked as a operator could do it using the GUI or the CLI.

This article shows you how use the xebialabs/xldeploy Puppet module.

The Production environment is based on 2 tomcats instances (tomcat1 & tomcat2) and a MySql database (dbprod) This information is configured in site.pp file: three modules are used xld-base, xld-tomcat, xld-mysql.

node 'tomcat1','tomcat2' {
$environment = "PuppetDemo"
include java
include xld-base
include xld-tomcat
}

node 'dbprod' {
$environment = "PuppetDemo"
include xld-base
include xld-mysql
}

XLD-Base module
This module manages the configuration of the node itself : it is a simple class

xldeploy_ci { "Infrastructure/$environment":
type => 'core.Directory',
rest_url => $xldeploy_url
}

xldeploy_ci { "Environments/$environment":
type => 'core.Directory',
rest_url => $xldeploy_url
}

xldeploy_ci { "Infrastructure/$environment/$fqdn":
type => "overthere.SshHost",
rest_url => $xldeploy_url,
properties => {
os => UNIX,
address => $ipaddress_eth1,
username => vagrant,
password => vagrant,
connectionType => INTERACTIVE_SUDO,
sudoUsername => $sudo_username,
stagingDirectoryPath => $staging_directory_path
},
}

xldeploy_ci {"Environments/$environment/App-$environment":
type => 'udm.Environment',
properties => { },
rest_url => $xldeploy_url
}

The xldeploy_ci resources used here will ensure:

    Infrastructure/$environment and Environments/$environment
  • the overthere ssh host is configured in the repository with the Infrastructure/$environment/$fqdn ID : the fully qualified domain name ($fqdn) and the IP address $ipaddress_eth1 are provided by the Puppet facts. The other parameters ($sudo_username, $staging_directory_path) are provided by the Hiera database.
  • the target environment Environments/$environment/App-$environment is created.

the $rest_url parameters is provided by the hiera configuration, it includes the address, the port, the credentianl and the context of the XL Deploy server. The value used here is : http://admin:[email protected]:4516/deployit

XLD-Tomcat module
This module manages the tomcat configuration and the information that need to be configured in XL Deploy.

The first part of the class is the configuration of the tomcat instance

include java

class { 'tomcat':
version => '7',
sources => true,
sources_src => 'file:/vagrant/tomcat'
}

tomcat::instance { 'appserver':
ensure => present,
server_port => $tomcat_port_mgt,
http_port => $tomcat_port_http,
ajp_port => $tomcat_port_ajp,
}

Then the configuration for XL Deploy repository:

xldeploy_ci { "Infrastructure/$environment/$fqdn/appserver-$hostname":
type => 'tomcat.Server',
properties => {
stopCommand => '/etc/init.d/tomcat-appserver stop',
startCommand => 'nohup /etc/init.d/tomcat-appserver start',
home => '/srv/tomcat/appserver',
stopWaitTime => 0,
startWaitTime => 10,
deploymentGroup => "$deployment_group",
},
rest_url => $xldeploy_url
}

xldeploy_ci { "Infrastructure/$environment/$fqdn/appserver-$hostname/$hostname.vh":
type => 'tomcat.VirtualHost',
properties => {
deploymentGroup => "$deployment_group",
},
rest_url => $xldeploy_url
}

The two xldeploy_ci resources configure the ‘tomcat.Server’ and the associated ‘tomcat.VirtualHost’ Configuration items. They share the same deployment group ($deployment_group) The ‘autorequire’ feature has been implements so it is not necessary to define explicitly ‘require’ between the 2 resources.

The module offers to define dictionaries, to populate them with values managed by Puppet (ex tomcat.http.port or environment name) and to associate them to environments.

xldeploy_ci { "Environments/$environment/$fqdn.dict":
type => "udm.Dictionary",
properties => {
entries => {
"log.RootLevel" => "ERROR",
"log.FilePath" => "/tmp/null",
"tomcat.port" => "$tomcat_port_http",
"tests2.ExecutedHttpRequestTest.url" => "http://localhost:{{tomcat.port}}/petclinic/index.jsp",
"tomcat.DataSource.username" => "scott",
"tomcat.DataSource.password" => "tiger",
"TITLE" => "$environment",
"tomcat.DataSource.driverClassName" => "com.mysql.jdbc.Driver",
"tomcat.DataSource.url" => "jdbc:mysql://localhost/{{tomcat.DataSource.context}}",
"tomcat.DataSource.context" => "petclinic",
"tests2.ExecutedHttpRequestTest.expectedResponseText" => "Home",
},
restrictToContainers => ["Infrastructure/$environment/$fqdn/appserver-$hostname/$hostname.vh", "Infrastructure/$environment/$fqdn/test-runner-$hostname", "Infrastructure/$environment/$fqdn/appserver-$hostname"],
},
rest_url => $xldeploy_url,
require => [Xldeploy_ci["Infrastructure/$environment/$fqdn/appserver-$hostname/$hostname.vh"],
Xldeploy_ci[ "Infrastructure/$environment/$fqdn/test-runner-$hostname"],
Xldeploy_ci["Infrastructure/$environment/$fqdn/appserver-$hostname"]],
}

Finally we gather all these containers and the dictionaries in the target enviroment:

xldeploy_environment_member { "Manage Tomcat members of Environments/$environment/App-$environment":
env => "Environments/$environment/App-$environment",
members => ["Infrastructure/$environment/$fqdn/appserver-$hostname/$hostname.vh", "Infrastructure/$environment/$fqdn/test-runner-$hostname", "Infrastructure/$environment/$fqdn/appserver-$hostname"],
dictionaries => ["Environments/$environment/$fqdn.dict"],
rest_url => $xldeploy_url,
}

Find the complete manifest here: https://github.com/xebialabs-community/xl-deploy-puppet-sample/blob/master/puppet/modules/xld-tomcat/manifests/init.pp

XLD-MySQL module
This module is designed as the previous one: one section to configure the database instance, the other to configure it in XL Deploy. Note the same parameters ($dbuser, $dbpasword and $dbname) are used to configure the database, the SqlContainer and the dictionary for the tomcat datasource. If the security team decides to change it, it’ve been defined in a single location and the information can be propagated to the node and the deployed application.

mysql::db { "$dbname":
user => $dbuser,
password => $dbpassword,
host => '%',
grant => ['all'],
}


xldeploy_ci { "Infrastructure/$environment/$fqdn/mysql-$dbname":
type => 'sql.MySqlClient',
properties => {
username => "$dbuser",
password => "$dbpassword",
databaseName => "$dbname",
mySqlHome => '/usr',
deploymentGroup => "1",
},
rest_url => $xldeploy_url,
}

xldeploy_ci { "Environments/$environment/App-db-$environment":
rest_url => $xldeploy_url,
type => 'udm.Dictionary',
properties => {
entries => {
'db.username' => "$dbuser",
'db.password' => "$dbpassword",
'db.name' => "$dbname",
'db.host' => "$ipaddress_eth1",
'db.url' => "jdbc:mysql://{{db.host}}:3306/{{db.name}}",
}},
}

Find the complete manifest file here: https://github.com/xebialabs-community/xl-deploy-puppet-sample/blob/master/puppet/modules/xld-mysql/manifests/init.pp)

The xl-deploy-puppet-module can manage roles, permission… check out the module documentation for the other features.

Wrap up
The integration between XL Deploy and Puppet applies the separation of concern principle, the one manages the provisioning, the other managed the application deployment and application configuration. The 2 solutions are model based : you describe the target and not the how to reach the target.

You can find all the described manifest files and the whole project based on Vagrant here: https://github.com/xeblialabs-community/xl-deploy-puppet-sample

The post XL Deploy & Puppet appeared first on XebiaLabs.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

Microservices Articles
When building large, cloud-based applications that operate at a high scale, it’s important to maintain a high availability and resilience to failures. In order to do that, you must be tolerant of failures, even in light of failures in other areas of your application. “Fly two mistakes high” is an old adage in the radio control airplane hobby. It means, fly high enough so that if you make a mistake, you can continue flying with room to still make mistakes. In his session at 18th Cloud Expo, Lee A...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
Lori MacVittie is a subject matter expert on emerging technology responsible for outbound evangelism across F5's entire product suite. MacVittie has extensive development and technical architecture experience in both high-tech and enterprise organizations, in addition to network and systems administration expertise. Prior to joining F5, MacVittie was an award-winning technology editor at Network Computing Magazine where she evaluated and tested application-focused technologies including app secu...
Containers and Kubernetes allow for code portability across on-premise VMs, bare metal, or multiple cloud provider environments. Yet, despite this portability promise, developers may include configuration and application definitions that constrain or even eliminate application portability. In this session we'll describe best practices for "configuration as code" in a Kubernetes environment. We will demonstrate how a properly constructed containerized app can be deployed to both Amazon and Azure ...
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the bene...
Using new techniques of information modeling, indexing, and processing, new cloud-based systems can support cloud-based workloads previously not possible for high-throughput insurance, banking, and case-based applications. In his session at 18th Cloud Expo, John Newton, CTO, Founder and Chairman of Alfresco, described how to scale cloud-based content management repositories to store, manage, and retrieve billions of documents and related information with fast and linear scalability. He addresse...
The now mainstream platform changes stemming from the first Internet boom brought many changes but didn’t really change the basic relationship between servers and the applications running on them. In fact, that was sort of the point. In his session at 18th Cloud Expo, Gordon Haff, senior cloud strategy marketing and evangelism manager at Red Hat, will discuss how today’s workloads require a new model and a new platform for development and execution. The platform must handle a wide range of rec...
SYS-CON Events announced today that DatacenterDynamics has been named “Media Sponsor” of SYS-CON's 18th International Cloud Expo, which will take place on June 7–9, 2016, at the Javits Center in New York City, NY. DatacenterDynamics is a brand of DCD Group, a global B2B media and publishing company that develops products to help senior professionals in the world's most ICT dependent organizations make risk-based infrastructure and capacity decisions.
Discussions of cloud computing have evolved in recent years from a focus on specific types of cloud, to a world of hybrid cloud, and to a world dominated by the APIs that make today's multi-cloud environments and hybrid clouds possible. In this Power Panel at 17th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the importance of customers being able to use the specific technologies they need, through environments and ecosystems that expose their APIs to make true ...
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...