| Both sides previous revision
Previous revision
Next revision
|
Previous revision
|
developers:jenkins:job_naming [2015/08/17 04:13]
nurupo |
developers:jenkins:job_naming [2017/10/08 07:57] (current)
nurupo [Jenkins Job Naming] Added Ubuntu Artful to the regex |
| ^( | ^( |
| ( # define some frequently used capture groups | ( # define some frequently used capture groups |
| (?!x)x # always false ("match only if there is no 'x' ahead and there is 'x' ahead") | (?!x)x # always false ("match only if there is no 'x' ahead and there is 'x' ahead") |
| (?<name>[^_]+) # group "name", name of the program/library | (?<name>[^_]+) # group "name", name of the program/library |
| (?<os>windows|linux|osx|android|illumos) # group "os", operating systems | (?<os>windows|linux|osx|android) # group "os", operating systems |
| (?<arch>x86-64|x86|arm64|arm) # group"arch", architectures | (?<arch>x86-64|x86|arm64|armhf|armel|multiarch) # group "arch", architectures |
| (?<buildtype>release|debug) # group"buildtype", build types | (?<buildtype>release|debug) # group "buildtype", build types |
| (?<linking>shared|static|mixed) # group "linking", library linking/build types | (?<linking>shared|static|mixed) # group "linking", library linking/build types |
| ) | ) |
| | | | |
| ( | ( |
| ((?<=linux_) # if there is "linux_" before here | ((?<=linux_) # if there is "linux_" before here |
| ( # then match | # then match |
| debian-8 | (deb|rpm)_ |
| )_ | (shared|static)_ |
| | ( |
| | ((?<=deb_shared_) # if there is "deb_shared_" before here |
| | # then match |
| | (jessie|stretch|buster|sid|xenial|yakkety|zesty|artful)_ |
| | ) |
| | | |
| | ((?<!deb_shared_) # if there is no "deb_shared_", don't match anything |
| | ) |
| | ) |
| ) | ) |
| | | | |
| ((?<!linux_)) # if there is no "linux_", don't match anything | ((?<!linux_) # if there is no "linux_", don't match anything |
| | ) |
| ) | ) |
| \g<arch>_(stable|nightly)_\g<buildtype> | \g<arch>_(stable|nightly)_\g<buildtype> |
| </code> | </code> |
| |
| You can test your job name against the regex using [[https://regex101.com/r/zJ9zD1/19|this regex101 save]]. | You can test your job name against the regex using [[https://regex101.com/r/zJ9zD1/32|this regex101 save]]. |
| |
| ===== Examples of valid job names ===== | ===== Examples of valid job names ===== |
| |
| | * ''antox_build_android_armhf_release'' |
| | * ''libtoxcore_analyze_cppcheck'' |
| * ''libtoxcore_analyze_scan-build'' | * ''libtoxcore_analyze_scan-build'' |
| * ''libtoxcore_build_linux_arm64_mixed_debug'' | * ''libtoxcore_build_linux_arm64_mixed_debug'' |
| * ''libtoxcore_build_linux_x86_64_mixed_debug'' | |
| * ''libtoxcore_build_windows_x86_shared_release'' | * ''libtoxcore_build_windows_x86_shared_release'' |
| * ''libtoxcore_src'' | * ''libtoxcore_src'' |
| * ''qTox_analyze_cppcheck'' | * ''qtox_build_osx_x86-64_debug'' |
| * ''qtox_pkg_linux_debian-8_x86-64_stable_release'' | * ''qtox_pkg_linux_deb_shared_jessie_x86-64_stable_release'' |
| * ''qtox_pkg_windows_x86-64_stable_release'' | * ''qtox_pkg_linux_deb_shared_yakkety_x86-64_stable_release'' |
| * ''utox_build_windows_x86_release'' | * ''qtox_pkg_linux_deb_static_armel_stable_debug'' |
| * ''utox_pkg_linux-debian-8_arm_stable_debug'' | * ''utox_build_windows_x86-64_release'' |
| * ''utox_pkg_osx_x86-64_nightly_debug'' | * ''utox_pkg_linux_rpm_shared_x86-64_nightly_release'' |
| * ''uTox_src'' | * ''utox_pkg_linux_rpm_static_x86-64_nightly_release'' |
| | * ''utox_pkg_osx_armhf_nightly_debug'' |
| | * ''utox_src'' |
| |
| Note that the name part of the job (''\g<name>'') can contain lower case letters, as well as upper case ones. The general rule of thumb is to use lowercase letters for everything unless you have a good reason to not to. The reason for that rule of thumb is that most jobs initially were named in lowercase only (i.e. ''libqt5'' instead of ''libQt5'', ''libffmpeg'', instead of ''libFFmpeg'', etc.), and sometime later this was enforced by adding lowercase restriction to the regex above, but qTox and uTox maintainers wanted to have "T" letter in their jobs names, so the lowercase-only restriction in regex was lifted because of that, which makes them an exception to the general rule. | Note that the name part of the job (''\g<name>'') can contain lower case letters, as well as upper case ones. The general rule of thumb is to use lowercase letters for everything unless you have a good reason to not to. The reason for that rule of thumb is that most jobs initially were named in lowercase only (i.e. ''libqt5'' instead of ''libQt5'', ''libffmpeg'', instead of ''libFFmpeg'', etc.), and sometime later this was enforced by adding lowercase restriction to the regex above, but qTox and uTox maintainers wanted to have "T" letter in their jobs names, so the lowercase-only restriction in regex was lifted because of that, which makes them an exception to the general rule. |
| ===== Usefulness of job naming ===== | ===== Usefulness of job naming ===== |
| |
| It's obvious that such job naming, which might seem a bit too explicit to some, explains right away just what a job is producing. We could have stopped on that fact, since it's quite an achievement by itself, but there is a smart application of it that you can find useful when managing big amount of jobs. | It's obvious that such job naming, which might seem a bit too explicit to some, makes it clear right away just what a job is producing. We could have stopped on that fact, since it's quite an achievement by itself, but there is a smart application of it that you can find useful when managing big amount of jobs. |
| |
| As an example, there are quite a few ''libqt5_build_*'' jobs in Jenkins: | As an example, there are quite a few ''libqt5_build_*'' jobs in Jenkins: |
| - ''libqt5_build_windows_x86_static_release'' | - ''libqt5_build_windows_x86_static_release'' |
| |
| Qt has many build flags that we might need to enable/disable which are almost identical to all of the builds, with just a few differences: static builds pass ''-static'' flag, shared builds pass ''-shared'' flag, release builds pass ''-release'' flag, debug builds pass ''-debug'' flag, Linux x86-64 builds pass ''-platform linux-g++-64'' flag, Linux x86 builds pass ''-platform linux-g++-32'' flag and so on, but the majority of flags is the same and if we were to edit some of the flags, it's highly likely to be one of the general flags, so it would need to be changed in all of the ''libqt5'' builds, which are... well, just too many. And if the change didn't succeed, one would need to fix the change, modifying all of the jobs again. It's a pretty cumbersome exercise to do. However, there is a solution to this. If you look closely enough, you will notice that all of the special flags that differentiate one job from another are summarized in the job name. In essence, job name tell us which flags to apply, which we can use to our advantage. Jenkins defines ''$(JOB_NAME)'' environment variable (along with [[https://build.tox.chat/env-vars.html/|many other]]), which we can use to write a single bash script that will build Qt differently based on nothing but job's name! You can write such a script, make a separate job that produces just that script, make all of ''libqt5_build_*'' be downstream projects of that job and make them call the script, so that whenever the script changes, all of ''libqt5_build_*'' are rebuilt using the new version of the script. | Qt has many build flags that we might need to enable/disable from time to time and those flags are almost identical for all of the builds, with just a few differences: static builds pass ''-static'' flag, shared builds pass ''-shared'' flag, release builds pass ''-release'' flag, debug builds pass ''-debug'' flag, Linux x86-64 builds pass ''-platform linux-g++-64'' flag, Linux x86 builds pass ''-platform linux-g++-32'' flag and so on, but the majority of flags is the same and if we were to edit some of the flags, it's highly likely to be one of the general flags, so it would need to be changed in all of the ''libqt5'' builds, which are... well, just too many. And if the change didn't succeed, one would need to fix the change, modifying all of the jobs again. It's a pretty cumbersome exercise to do. However, there is a solution to this. If you look closely enough, you will notice that all of the special flags that differentiate one job from another are summarized in the job name. In essence, job name tell us which flags to apply, which we can use to our advantage. Jenkins defines ''$(JOB_NAME)'' environment variable (along with [[https://build.tox.chat/env-vars.html/|many other]]), which we can use to write a single bash script that will build Qt differently based on nothing but job's name! You can write such a script, make a separate job that produces just that script, make all of ''libqt5_build_*'' be downstream projects of that job and make them call the script, so that whenever the script changes, all of ''libqt5_build_*'' are rebuilt using the new version of the script. |
| |
| Here is an example of creating such a script in a Jenkins job using bash's [[http://tldp.org/LDP/abs/html/here-docs.html|Here Documents]], which you can use as a template for creating your own once-script-rules-them-all job: | Here is an example of creating such a script in a Jenkins job using bash's [[http://tldp.org/LDP/abs/html/here-docs.html|Here Documents]], which you can use as a template for creating your own once-script-rules-them-all job: |
| then | then |
| |
| if [[ ${JOB_NAME} == *"x86_64"* ]] | if [[ ${JOB_NAME} == *"x86-64"* ]] |
| then | then |
| QT_CONFIGURE_OPTIONS="-device-option CROSS_COMPILE=x86_64-w64-mingw32- $QT_CONFIGURE_OPTIONS" | QT_CONFIGURE_OPTIONS="-device-option CROSS_COMPILE=x86_64-w64-mingw32- $QT_CONFIGURE_OPTIONS" |
| | |
| else | else |
| echo "Error: Support only x86 and x86_64 architectires on Windows. Exiting." | echo "Error: Support only x86 and x86-64 architectires on Windows. Exiting." |
| exit 1 | exit 1 |
| fi | fi |
| then | then |
| | |
| if [[ ${JOB_NAME} == *"x86_64"* ]] | if [[ ${JOB_NAME} == *"x86-64"* ]] |
| then | then |
| QT_CONFIGURE_OPTIONS="-platform linux-g++-64 $QT_CONFIGURE_OPTIONS" | QT_CONFIGURE_OPTIONS="-platform linux-g++-64 $QT_CONFIGURE_OPTIONS" |
| | |
| else | else |
| echo "Error: Support only x86 and x86_64 architectires on Linux. Exiting." | echo "Error: Support only x86 and x86-64 architectires on Linux. Exiting." |
| exit 1 | exit 1 |
| fi | fi |
