[{"content":"Introduction\rJenkins is one of the most popular and powerful automation tools, widely used for integrating and deploying software projects continuously. While it\u0026rsquo;s possible to quickly start Jenkins using a simple Docker command, this setup looks like \u0026ldquo;Hello World\u0026rdquo; in any programming language: useful for begginers, but insufficient for production environments.\nIn this post, we go beyond the basics. The goal here is to guide you with some techniques for a (almost) robust and scalable Jenkins architecture using Docker and Docker Compose. We\u0026rsquo;ll explore fundamental concepts like machine isolation and set up an environment near real world scenario. By the end, you\u0026rsquo;ll have a solid structure ready to be adapted and expanded according to your needs.\nIf you\u0026rsquo;re ready to move from begginer to professional CI/CD environment, read on!\n\u0026ldquo;Environment near real world scenario\u0026rdquo; does not mean production environment. Use this post as a reference, not a production ready solution.\rPrerequisites\rBefore diving into the technical details, it\u0026rsquo;s essential to ensure that some prerequisites are met:\nLinux (In my case, I use Windows as main SO, so I need to use some virtualizer like WSL, Virtualbox, VmWare, etc.) Docker Key Concepts\rWith the prerequisites defined, we can now explore some key concepts that will guide the Jenkins setup.\nIsolation\rThe first best practice for using Jenkins is not to run everything on the same machine. Both for security and scalability reasons, it is recommended to use at least two machines.\nThe first machine is what we usually call the controller. It is the main machine, primarily responsible for the interface, settings, and job execution control.\nThe other machines are called nodes. These machines are responsible for executing all the jobs.\nMachine Configuration\rBoth the controller and the nodes need to be configured within an operating system.\nInstalling prerequisites, creating directories, and setting user permissions are some examples. Additionally, we will take this opportunity to perform some Jenkins configurations that will only take effect on startup or restart.\nWe will use the Jenkins images and Docker Compose for this step.\nJenkins Configuration\rWith Jenkins running, we need to configure the environment to work properly:\nAccess credentials Configuration of communication between the controller and the node These configurations will not be done manually. We will use the Configuration as Code plugin to automate this process.\nBasic Structure\rWith the concepts understood, it’s time to start defining the basic structure of the project.\nThis is the basic structure. This model will be used in future posts.\n. └── demo/ ├── casc/ # Jenkins configuration as code │ └── *.yaml ├── plugins/ │ └── plugins.txt # Plugin list ├── scripts/ │ └── *.groovy # Jenkins initialization scripts ├── ssh # Public and private keys ├── docker-compose.yaml ├── Dockerfile ├── start.sh └── stop.sh Dockerfile\rWithin this structure, the Dockerfile plays a central role. Let\u0026rsquo;s examine it in detail:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 FROM jenkins/jenkins:lts RUN mkdir -p $JENKINS_HOME/.ssh RUN chown -R jenkins:jenkins $JENKINS_HOME/.ssh RUN touch $JENKINS_HOME/.ssh/known_hosts RUN chmod 600 $JENKINS_HOME/.ssh/known_hosts COPY --chown=jenkins:jenkins ./ssh/jenkins_key $JENKINS_HOME/.ssh/jenkins_key RUN chmod 600 $JENKINS_HOME/.ssh/jenkins_key COPY --chown=jenkins:jenkins ./scripts/ /usr/share/jenkins/ref/init.groovy.d/ COPY --chown=jenkins:jenkins ./plugins/plugins.txt /usr/share/jenkins/ref/plugins.txt RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt RUN mkdir -p $JENKINS_HOME/casc_configs RUN chown -R jenkins:jenkins $JENKINS_HOME/casc_configs COPY --chown=jenkins:jenkins ./casc/* $JENKINS_HOME/casc_configs/ ENV CASC_JENKINS_CONFIG=\u0026#34;$JENKINS_HOME/casc_configs/\u0026#34; ENV JAVA_OPTS=\u0026#34;-Djenkins.install.runSetupWizard=false\u0026#34; A summary of the Dockerfile content:\nlines 3 to 8: Creation of the .ssh directory and copying of the private key stored in the project\u0026rsquo;s ssh folder. line 10: Copying of Groovy scripts stored in the project\u0026rsquo;s scripts folder. lines 12 and 13: Copying the plugin list and installing the plugins in the image. lines 15 to 17: Creation of the JCasC plugin folder and copying the configuration files from the project\u0026rsquo;s casc folder (Configuration as Code). line 20: Sets the environment variable CASC_JENKINS_CONFIG to configure the default folder where Jenkins will retrieve these files. line 21: Configures the environment variable JAVA_OPTS to skip the installation Wizard. docker-compose.yaml\rAfter configuring the Dockerfile, we need to orchestrate the services with docker-compose.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 version: \u0026#39;3.7\u0026#39; volumes: jenkins_data: name: jenkins_data services: jenkins: container_name: jenkins_master build: . ports: - 8080:8080 - 50000:50000 volumes: - jenkins_data:/var/jenkins_home/ - /var/run/docker.sock:/var/run/docker.sock ssh-agent: container_name: jenkins_agent image: jenkins/ssh-agent environment: - JENKINS_AGENT_SSH_PUBKEY=${JENKINS_AGENT_SSH_PUB} depends_on: - jenkins There is no secret to docker-compose. The jenkins service runs the official Jenkins image, with the main ports mapped, and the ssh-agent service will be the execution node for the pipelines.\nIn line 19, we configure the environment variable JENKINS_AGENT_SSH_PUBKEY to inform the generated public key.\nplugins.txt\rNow that the services are configured, let\u0026rsquo;s define the essential plugins in the plugins.txt file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 cloudbees-folder build-timeout credentials timestamper ws-cleanup pipeline-milestone-step pipeline-input-step pipeline-stage-step pipeline-graph-analysis pipeline-rest-api pipeline-stage-view pipeline-build-step pipeline-model-api pipeline-model-extensions pipeline-stage-tags-metadata pipeline-model-definition ssh-credentials ssh-slaves config-file-provider workflow-aggregator configuration-as-code executors.groovy\rFollowing the recommendations at the beginning of the article, it is necessary to adjust the Jenkins executors, which we will do in the executors.groovy script:\n1 2 import jenkins.model.* Jenkins.instance.setNumExecutors(0) // Recommended to not run builds on the built-in node ssh-agent.yaml\rFinally, we need to configure secure communication between the controller and the nodes, using ssh-agent.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 credentials: system: domainCredentials: - credentials: - basicSSHUserPrivateKey: scope: SYSTEM id: ssh_agent_key username: description: \u0026#34;SSH passphrase with private key file. Private key provided\u0026#34; privateKeySource: directEntry: privateKey: \u0026#34;${readFile:${JENKINS_HOME}/.ssh/jenkins_key}\u0026#34; jenkins: nodes: - permanent: launcher: ssh: credentialsId: \u0026#34;ssh_agent_key\u0026#34; host: \u0026#34;jenkins_agent\u0026#34; port: 22 sshHostKeyVerificationStrategy: \u0026#34;nonVerifyingKeyVerificationStrategy\u0026#34; name: \u0026#34;jenkins_agent\u0026#34; numExecutors: 2 remoteFS: \u0026#34;/tmp\u0026#34; retentionStrategy: \u0026#34;always\u0026#34; Here, we configure Jenkins using the configuration as code plugin:\nLines 1 to 12: Creation of a Jenkins credential of type SSH Username with private key with the private key. Lines 13 to 25: We create the node configuration. Start up\rWith all the configurations in place, it’s time to put everything into practice and start the environment. We’ll use the start.sh script to automate the execution of docker-compose (especially useful for repeated executions):\n1 2 3 4 5 6 7 8 9 10 #!/bin/sh work_path=$(pwd) echo $work_path sh $work_path/stop.sh ssh-keygen -t rsa -f $work_path/ssh/jenkins_key -N \u0026#34;\u0026#34; -C \u0026#34;\u0026#34; export JENKINS_AGENT_SSH_PUB=$(cat $work_path/ssh/jenkins_key.pub) echo $JENKINS_AGENT_SSH_PUB docker-compose up --build -d In summary, the start.sh script:\nExecutes the stop.sh script to remove any existing environment. Generates the ssh key for secure communication between the jenkins and ssh-agent containers. Exports the environment variable JENKINS_AGENT_SSH_PUB. Builds and starts the environment. Conclusion\rIn this article, we explored some essential concepts for a robust Jenkins setup and then proceeded with the actual configuration.\nStarting with the environment setup, we used the Dockerfile and docker-compose.yaml to define and orchestrate the environment\u0026rsquo;s services. We then configured Jenkins\u0026rsquo; essential features, such as plugins and executors, using the plugins.txt and executors.groovy files.\nFinally, we used a script (start.sh) to automate the execution of all these steps.\nThe final project will be available in the repository repo-link.\nIn future articles, we will add more steps and configurations for Jenkins. If you\u0026rsquo;re interested, stay tuned for updates.\nPhoto by Chilli Charlie on Unsplash\n","date":"2024-08-15T00:00:00Z","image":"https://www.sitedoricardo.com.br/p/getting-started-with-jenkins-and-docker/chilli-charlie-6Lh4FyFdCxI-unsplash_hu5459c0360c2b0cb7a147d2df0eb350ca_855871_120x120_fill_q75_box_smart1.jpg","permalink":"https://www.sitedoricardo.com.br/p/getting-started-with-jenkins-and-docker/","title":"Getting started with Jenkins and Docker"},{"content":"Hello World!!!\rIf you read this, the site is live. 👍\n🎉\n","date":"2019-09-12T00:00:00Z","image":"https://www.sitedoricardo.com.br/p/im-alived/excavator_hua543d5175ea7410e792fb1275b608214_681042_120x120_fill_q75_box_smart1.jpg","permalink":"https://www.sitedoricardo.com.br/p/im-alived/","title":"I'm alived!"}]