Project

General

Profile

Actions

Infrastructure #3

closed

The pipeline for making BlackBox Setup file

Added by I. Denisov over 11 years ago. Updated over 9 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
-
Start date:
07/08/2014
Due date:
% Done:

100%

Estimated time:

Description

The binary setup file of BlackBox should be automatically generated on the server each time some changes made in repository.

The console compiler (see issue #2) allows to prepare executables from the sources. The next step is the Setup file compilation. Oberon microsystems used Inno Setup application for making the binary for distribution. Mark Frei shared the script which they had been using.

We can modify this script for our porpoises, add to the developers repository and use in maintaining pipeline on the server.


Files

BlackBox.iss (123 KB) BlackBox.iss OmincOriginalCompilation I. Denisov, 07/08/2014 10:05 PM
BlackBox.iss (123 KB) BlackBox.iss InnoSetupBlackBoxCompilation I. Denisov, 07/08/2014 10:07 PM
Actions #1

Updated by I. Denisov over 11 years ago

I modified this script:

1. The compression made
2. The organisation and URL changed

Actions #2

Updated by I. Denisov over 11 years ago

The resulted builds will be stores here now

http://blackboxframework.org/dev/

The console compiler and script for preparation of setup added to the development brunch
commit:4e0f4b1b40aedbabc85756260f843340b925af18

Actions #3

Updated by I. Denisov over 11 years ago

  • Blocked by Infrastructure #2: Command line compiler required for building BlackBox from sources added
Actions #4

Updated by I. Denisov over 11 years ago

  • Blocked by Feature #6: Version and Copyright in properties of BlackBox setup file and BlackBox.exe added
Actions #6

Updated by I. Denisov over 11 years ago

  • % Done changed from 20 to 80

I made the webhook, which runs each time somebody push to the GitHub repository.

<?php
if (isset($_POST['payload'])) {
    echo shell_exec("echo 'lock' > /home/git/bbcb.git.lock");
    file_put_contents("/home/git/bbcb.git.post", print_r($_POST, true));
    echo shell_exec("cd /home/git/bbcb.git && git fetch --all -p 2>&1");
    echo shell_exec("rm /home/git/bbcb.git.lock");
} else {
    echo "This is GitHub hook for update local repository.";
}
?>

I made the python script for making the Setup file of BlackBox:

#!/usr/bin/python

from subprocess import Popen, PIPE
import sys, datetime

# this files contains hashes of last commits in master and development branches
DEVHASH = open("/var/www/tribiq/build/devHash", "r+")
STABLEHASH = open("/var/www/tribiq/build/stableHash", "r+")

process1 = Popen("cd /home/git/bbcb.git && git log", stdout=PIPE, stderr=PIPE, shell=True)
(stdout1, stderr1) = process1.communicate()
if stderr1 == "":
    hash1 = stdout1.split("\n")[0].split(" ")[1]
    if STABLEHASH.readline().strip() == hash1:
        rebuildStable = False
        #print "No new commits in 'master'<br>" 
    else:
        rebuildStable = True
        #print "New commit in 'master', need to run build pipeline<br>" 
else:
    print stderr1

process2 = Popen("cd /home/git/bbcb.git && git log development", stdout=PIPE, stderr=PIPE, shell=True)
(stdout2, stderr2) = process2.communicate()
if stderr2 == "":
    hash2 = stdout2.split("\n")[0].split(" ")[1]
    if DEVHASH.readline().strip() == hash2:
        rebuildDev = False
        #print "No new commits in 'developent'<br>" 
    else:
        rebuildDev = True
        #print "New commit in 'development', need to run build pipeline<br>" 
else:
    print stderr2

def log(out, err):
    if out != "":    
        for line1 in out.split("\n"):
            print line1 + "<br>" 
    if err != "":
        for line2 in err.split("\n"):
            print line2 + "<br>" 

#### BUILDING STABLE

if rebuildStable:
    print "Building from 'master' at " + datetime.datetime.now().isoformat() + "<br>" 
    print "The build pipeline for 'master' brunch is not developed..." 
    # STABLEHASH.seek(0)
    # STABLEHASH.write(hash1)
    # STABLEHASH.truncate()

#### BUILDING DEVELOPMENT
if rebuildDev:

    print "<br><b>Building from 'development' at " + datetime.datetime.now().isoformat() + "</b><br>" 
    print "Trying to check repository state...<br>" 
    try:
        f = open('/home/git/bbcb.git.lock', 'r')
    except IOError:
        print "Repository is not locked. Continuing.<br>" 
    else:
        print "Repository is locked (possibly because of sync process).<br>Aborting.<br>" 
        f.close()
        sys.exit()

    print "Trying to clone repository into temporary folder...<br>" 

    process1 = Popen("cd /var/www/tribiq/build && git clone /home/git/bbcb.git bb", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout1, stderr1) = process1.communicate()
    log(stdout1, stderr1)

    print "Checking out development...<br>" 
    process2 = Popen("cd /var/www/tribiq/build/bb && git checkout development", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout2, stderr2) = process2.communicate()
    log(stdout2, stderr2)

    print "Building BlackBox...<br>" 
    process3 = Popen("cd /var/www/tribiq/build/bb && /usr/local/bin/wine dev0.exe < build.txt 2>&1", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout3, stderr3) = process3.communicate()
    log(stdout3, stderr3)

    print "Building Setup File...<br>" 
    process4 = Popen("cd /var/www/tribiq/build/bb && mv Code System/ && mv Sym System/ && cd appbuild && /usr/local/bin/iscc - < ./BlackBox.iss", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout4, stderr4) = process4.communicate()
    log(stdout4, stderr4)

    IN = open("/var/www/tribiq/build/bb/appbuild/BlackBox.iss")

    for line in IN:
        if line.split("=")[0] == "AppVersion":
            version =  line.split("=")[1].strip()
    print "The version of build: " + version + "<br>" 

    print "Moving Setup File to dev folder...<br>" 
    process5 = Popen("mv /var/www/tribiq/build/bb/appbuild/Output/setup.exe /var/www/tribiq/dev/blackbox-" + version + "-setup.exe", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout5, stderr5) = process5.communicate()
    log(stdout5, stderr5)

    if stderr5 == "":
        print "Hash for 'development' branch updated<br>" 
        DEVHASH.seek(0)
        DEVHASH.write(hash2)
        DEVHASH.truncate()

    print "Cleaning...<br>" 
    process6 = Popen("cd /var/www/tribiq/build && rm bb -R", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout6, stderr6) = process6.communicate()
    log(stdout6, stderr6)

It runs each 5 minutes with cron.d service:

*/5 * * * * www-data /var/www/tribiq/build/build.py >> /var/www/tribiq/dev/buildlog.html 2>&1
Actions #7

Updated by I. Denisov about 11 years ago

New build script with zip support and incremental build number.

#!/usr/bin/python

from subprocess import Popen, PIPE
import sys, datetime, fileinput

buildDir = "/var/www/tribiq/build" 
localRepository = "/var/www/git/bbcb.git" 
devDir = "/var/www/tribiq/dev" 
stableDir = "/var/www/tribiq/stable" 
wine = "/usr/local/bin/wine" 
iscc = "/usr/local/bin/iscc" 

# this files contains hashes of last commits in master and development branches
DEVHASH = open(buildDir + "/devHash", "r+")
STABLEHASH = open(buildDir + "/stableHash", "r+")

process1 = Popen("cd " + localRepository + " && git log", stdout=PIPE, stderr=PIPE, shell=True)
(stdout1, stderr1) = process1.communicate()
if stderr1 == "":
    hash1 = stdout1.split("\n")[0].split(" ")[1]
    if STABLEHASH.readline().strip() == hash1:
        rebuildStable = False
        #print "No new commits in 'master'<br>" 
    else:
        rebuildStable = True
        print "New commit in 'master', need to run build pipeline<br>" 
else:
    print stderr1

process2 = Popen("cd " + localRepository + " && git log development", stdout=PIPE, stderr=PIPE, shell=True)
(stdout2, stderr2) = process2.communicate()
if stderr2 == "":
    hash2 = stdout2.split("\n")[0].split(" ")[1]
    if DEVHASH.readline().strip() == hash2:
        rebuildDev = False
        # print "No new commits in 'developent'<br>" 
    else:
        rebuildDev = True
        print "New commit in 'development', need to run build pipeline<br>" 
else:
    print stderr2

def log(out, err):
    if out != "":    
        for line1 in out.split("\n"):
            print line1 + "<br>" 
    if err != "":
        for line2 in err.split("\n"):
            print line2 + "<br>" 

#### BUILDING STABLE
if rebuildStable:
    print "Building from 'master' at " + datetime.datetime.now().isoformat() + "<br>" 
    print "The build pipeline for 'master' branch is not developed..." 
    # STABLEHASH.seek(0)
    # STABLEHASH.write(hash1)
    # STABLEHASH.truncate()

#### BUILDING DEVELOPMENT
if rebuildDev:

    print "<br><b>Building from 'development' at " + datetime.datetime.now().isoformat() + "</b><br>" 
    print "Trying to check repository state...<br>" 
    try:
        f = open(localRepository + ".lock", "r")
    except IOError:
        print "Repository is not locked. Continuing.<br>" 
    else:
        print "Repository is locked (possibly because of sync process).<br>Aborting.<br>" 
        f.close()
        sys.exit()

    print "Trying to clone repository into temporary folder...<br>" 

    process1 = Popen("cd " + buildDir + " && git clone " + localRepository + " bb", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout1, stderr1) = process1.communicate()
    log(stdout1, stderr1)

    print "Checking out development...<br>" 
    process2 = Popen("cd " + buildDir + "/bb && git checkout development", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout2, stderr2) = process2.communicate()
    log(stdout2, stderr2)

    print "Building BlackBox...<br>" 
    process3 = Popen("cd " + buildDir + "/bb && " + wine + " dev0.exe < build.txt 2>&1", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout3, stderr3) = process3.communicate()
    log(stdout3, stderr3)

    NUMBER = open("number", "r+")
    buildNum = int(NUMBER.readline().strip())
    IN = open("bb/appbuild/BlackBox.iss", "r")
    lines = IN.readlines()
    IN.close()
    OUT = open("bb/appbuild/BlackBox.iss", "w")
    n = 0
    for line in lines:
        if n == 1:
            version = line.split('"')[1]
        n = n + 1
        if line[:18] == "VersionInfoVersion":
            print >> OUT, line.strip() + str(buildNum).zfill(3)
        else:
            print >> OUT, line,
    OUT.close()

    version = version + "." + str(buildNum).zfill(3)

    print "The version of build: " + version + "<br>" 

    print "Building Setup File...<br>" 
    process4 = Popen("cd " + buildDir + "/bb && mv Code System/ && mv Sym System/ && cd appbuild && " + iscc + " - < ./BlackBox.iss", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout4, stderr4) = process4.communicate()
    log(stdout4, stderr4)

    print "Moving Setup File to dev folder...<br>" 
    process5 = Popen("mv " + buildDir + "/bb/appbuild/Output/setup.exe " + devDir + "/blackbox-" + version + "-setup.exe", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout5, stderr5) = process5.communicate()
    log(stdout5, stderr5)

    if stderr5 == "":
        print "Hash for 'development' branch updated<br>" 
        DEVHASH.seek(0)
        DEVHASH.write(hash2)
        DEVHASH.truncate()
        print "Build number updated to " + str(buildNum + 1) + "<br>" 
        NUMBER.seek(0)
        NUMBER.write(str(buildNum+1))
        NUMBER.truncate()

    print "Zipping...<br>" 
    process6 = Popen("cd " + buildDir + "/bb && rm -R *.txt appbuild dev0.exe && zip -r ../../dev/blackbox-" + version + ".zip *", stdout=PIPE, stderr=PIPE, shell=True)
    (stdout6, stderr6) = process6.communicate()
    log(stdout6, stderr6)

    print "Cleaning...<br>" 
        process7 = Popen("cd " + buildDir + " && rm bb -R", stdout=PIPE, stderr=PIPE, shell=True)
        (stdout7, stderr7) = process7.communicate()
        log(stdout6, stderr7)
Actions #8

Updated by I. Denisov over 9 years ago

  • Tracker changed from 3 to Feature
Actions #9

Updated by I. Denisov over 9 years ago

  • Tracker changed from Feature to Infrastructure
  • % Done changed from 80 to 100
Actions #10

Updated by I. Denisov over 9 years ago

  • Project changed from 1 to BlackBox Component Builder
  • Status changed from In Progress to Closed
Actions

Also available in: Atom PDF