Share files between different agent workspace in the same Jenkins Pipeline via stash|unstash

 


Share files between different agent workspace in the same Jenkins Pipeline via stash|unstash
Photo by The Halal Design Studio / Unsplash

I was working on a legacy app recently. The legacy app has angular as its frontend and spring boot app as its backend. Originally the frontend build outputs are committed in the app's git repository and packaged by spring boot by copying them to the ${project.basedir}/target/classes/static/ class path(since content under the folder will be served as static resources by spring boot). And we are using Jenkins pipeline to run the build for this app. Hence the pipeline only use a java jenkins slave to compile and package the spring boot app. But that is not the right way, isn't it?

We should run both build the pipeline for frontend angular app and backend sprint boot app. But an angular app required a node slave to build it out. We don't have a Jenkins slave that has both java and node installed. Then issue comes: How to build java and javascript apps in a single Jenkins pipeline?

Then I found Pipeline allows utilizing multiple agents in the Jenkins environment from within the sameJenkinsfile, which can helpful for more advanced use-cases such as executing builds/tests across multiple platforms.

utilizing multiple agents in Pipeline

Good. We could use node agent to build the angular app and use java agent to compile the spring boot app.

    agent {
        label 'maven' # agent can build java app
    }
    stages {
        stage('Build UI') {
            agent {
                label 'node8' # agent can build javascript app
            }
            steps {
                sh 'npm install'
                sh 'run build'
            }
        }
        stage('Compile') {
            // Build the code to generate the spring-boot jar
            steps {
                sh "mvn -B clean install -f ${POM_FILE}"
            }
        }
    }

The above snippet works well, both frontend and backend got built out. I encountered another challenge: Each agent will run the steps in its own container, which means we are not able to access the dist build out from node agent in the maven agent, since they are using different workspaces.

Share files between agents

While I was searching a proper way to address the above concern, I found stash and unstash directives:

stash: Stash some files to be used later in the build
unstash: Restore files previously stashed

Jenkinsfile

pipeline {
    agent {
        label 'maven'
    }
    environment {
        POM_FILE 'server/pom.xml'
    }
    stages {
        stage('Build UI') {
            agent {
                label 'node8'
            }
            steps {
                sh 'npm --prefix=ui install'
                sh 'npm --prefix=ui run build'
                stash includes: 'ui/dist/**', name: 'uidist'
            }
        }
        stage('Compile') {
            // Build the code to generate the spring-boot jar
            steps {
                sh "mvn -B clean install -f ${POM_FILE}"
            }
        }
        stage('Package') {
            // Build the code to generate the spring-boot jar
            steps {
                unstash "uidist"
                sh "ls -all ui/dist"
                sh "mvn -DskipTests=true -B package -f ${POM_FILE}"
            }
        }
    }
}
<plugin>
<artifactId>maven-resources-plugin</artifactId>
<executions>
<execution>
<id>copy-resources</id>
<phase>validate</phase>
<goals>
<goal>copy-resources</goal>
</goals>
<configuration>
<outputDirectory>${project.basedir}/target/classes/static/</outputDirectory>
<resources>
<resource>
<directory>../ui/dist</directory>
</resource>
</resources>
</configuration>
</execution>
</executions>
</plugin>

References:


https://docs.cloudbees.com/docs/admin-resources/latest/automating-with-jenkinsfile/using-multiple-agents

What does a ws() block do in Jenkins?
I’m trying to convert few Jenkinsfiles from Scripted Pipeline to Declarative Pipeline. I have a block like this in a Jenkinsfile: ws(”/path/to/dir”) { // do stuff} I wonder what does it do e...
How to re-use previously created workspace across stages
I am facing an issue where I have two stages defined in my pipeline that both are run on the same node and need to be run in the same workspace. The first of these stages runs on my master node
Aggregating results of downstream parameterised jobs in Jenkins
I have a Jenkins Build job which triggers multiple Test jobs with the test name as a parameter using the Jenkins Parameterized Trigger Plugin. This kicks off a number of test builds on multiple exe...

No comments:

Post a Comment

fixed: embedded-redis: Unable to run on macOS Sonoma

Issue you might see below error while trying to run embedded-redis for your testing on your macOS after you upgrade to Sonoma. java.la...