Opened 3 years ago

Last modified 4 days ago

#915 assigned task

Rethink BCCD build process

Reported by: skylar Owned by: skylar
Priority: major Milestone: 3.4.0-build_ng
Component: Both Version:
Keywords: Cc:
Blocked By: Blocking:
Estimated Hours: Total Hours: 0

Description

Goals:

  • Make dependencies in the build process more transparent (or remove them entirely)
  • Make BCCD more of a package/channel on top of Debian

Change History (170)

comment:1 Changed 3 years ago by skylar

In 5099//cluster/svnroot:

adding documentation pointers re #915

comment:2 Changed 3 years ago by skylar

In 5116//cluster/svnroot:

merge addresses #872, #892, #902, #905, #906, #908, #915, #919, #920, #922

comment:3 Changed 3 years ago by skylar

In 5145//cluster/svnroot:

adding docs re #915

comment:5 Changed 3 years ago by skylar

comment:6 Changed 3 years ago by skylar

In 5240//cluster/svnroot:

merge addresses #896, #898, #915, #946, #949, #950

comment:7 Changed 3 years ago by skylar

In 5301//cluster/svnroot:

more docs re #915

comment:8 Changed 3 years ago by skylar

In 5302//cluster/svnroot:

more docs re #915

comment:9 Changed 3 years ago by skylar

In 5303//cluster/svnroot:

more docs re #915

comment:10 Changed 3 years ago by skylar

In 5330//cluster/svnroot:

merge addresses #910, #915, #930, #951, #956, #957, #958, #959, #960

comment:11 Changed 3 years ago by skylar

trying this:

lb config --architecture amd64 --binary-images iso --apt aptitude --distribution sid --linux-packages aptitude

lb build

Last edited 3 years ago by skylar (previous) (diff)

comment:12 Changed 3 years ago by skylar

above fails with error about missing aptitude

trying with apt:

lb config --architecture amd64 --binary-images iso --distribution sid

lb build

comment:13 Changed 3 years ago by skylar

and now trying to disable systemd:

lb config --architecture amd64 --binary-images iso --distribution sid --initsystem sysvinit

comment:14 Changed 3 years ago by skylar

errors:

dpkg: dependency problems prevent removal of systemd-sysv:

init depends on systemd-sysv | sysvinit-core | upstart; however:

Package systemd-sysv is to be removed. Package sysvinit-core is not installed. Package upstart is not installed.

dpkg: error processing package systemd-sysv (--purge):

dependency problems - not removing

dpkg: dependency problems prevent removal of systemd:

systemd-sysv depends on systemd.

dpkg: error processing package systemd (--purge):

dependency problems - not removing

Errors were encountered while processing:

systemd-sysv systemd

comment:15 Changed 3 years ago by skylar

lb config --architecture amd64 --binary-images iso --distribution sid --initsystem sysvinit --archive-areas "main contrib non-free"

comment:16 Changed 3 years ago by skylar

removing systemd packages from config/chroot_local-packageslists

comment:17 Changed 3 years ago by skylar

going to try using systemd to see if it works at all, then remove it later

lb config --architecture amd64 --binary-images iso --distribution sid --initsystem systemd --archive-areas "main contrib non-free"

comment:18 Changed 3 years ago by skylar

trying in ubuntu:

sudo lb config --architecture amd64 --binary-images iso --distribution trusty --initsystem systemd

leaving off --archive-areas since it seems ubuntu does not have those

Last edited 3 years ago by skylar (previous) (diff)

comment:19 Changed 3 years ago by skylar

now trying in ubuntu:

lb_config --architecture amd64 --binary-images iso --distribution trusty --initsystem systemd --mode ubuntu

comment:20 Changed 3 years ago by skylar

and now

lb config --binary-images iso

comment:21 Changed 3 years ago by skylar

failing on finding live-config and live-config-systemd

comment:22 Changed 3 years ago by skylar

trying w/ 4.0 alpha install on bigfe and almaren

comment:23 Changed 3 years ago by skylar

might need to use hdd for USB sticks:

lb config --architecture amd64 --binary-images hdd --distribution sid --initsystem systemd --archive-areas "main contrib non-free"

comment:24 Changed 3 years ago by skylar

building on ubuntu is a non-starting due to hard-coded isolinux dependency (ubuntu does not have isolinux)

building ubuntu on debian might work though

comment:25 Changed 3 years ago by skylar

In 5340//cluster/svnroot:

branching for build process improvement re #915

comment:26 Changed 3 years ago by skylar

In 5341//cluster/svnroot:

trying python configparser re #915

comment:27 Changed 3 years ago by skylar

In 5342//cluster/svnroot:

wrong directory re #915

comment:28 Changed 3 years ago by skylar

In 5343//cluster/svnroot:

trying python configparser re #915

comment:29 Changed 3 years ago by skylar

In 5344//cluster/svnroot:

argument parsing re #915

comment:30 Changed 3 years ago by skylar

In 5345//cluster/svnroot:

move stuff to main re #915

comment:31 Changed 3 years ago by skylar

todo:

find module for logging to console

comment:32 Changed 3 years ago by skylar

looks like logging will do the trick

comment:33 Changed 3 years ago by skylar

In 5348//cluster/svnroot:

adding logger re #915

comment:34 Changed 3 years ago by skylar

In 5349//cluster/svnroot:

test for Jenkins re #915

comment:35 Changed 3 years ago by skylar

In 5350//cluster/svnroot:

merge addresses #915

comment:36 Changed 3 years ago by skylar

In 5351//cluster/svnroot:

rolling back re #915 for build_ng branch

comment:37 Changed 3 years ago by skylar

In 5352//cluster/svnroot:

adding arch as a required argument re #915

comment:38 Changed 3 years ago by skylar

In 5353//cluster/svnroot:

support any python 3 re #915

comment:39 Changed 3 years ago by skylar

In 5354//cluster/svnroot:

support pulling svnrev from command line re #915

comment:40 Changed 3 years ago by skylar

In 5355//cluster/svnroot:

allow setting log level from command line re #915

comment:41 Changed 3 years ago by skylar

In 5356//cluster/svnroot:

testing debootstrap re #915

comment:42 Changed 3 years ago by skylar

In 5357//cluster/svnroot:

place debootstrap into subdir of tempdir re #915

comment:43 Changed 3 years ago by skylar

In 5358//cluster/svnroot:

test install_packages re #915

comment:44 Changed 3 years ago by skylar

In 5364//cluster/svnroot:

adding package control re #915

comment:45 Changed 3 years ago by skylar

In 5366//cluster/svnroot:

fixing XOpenDisplay for Life re #915

comment:46 Changed 3 years ago by skylar

In 5368//cluster/svnroot:

fixing x11 for remaining modules re #915

comment:47 Changed 3 years ago by skylar

In 5369//cluster/svnroot:

better comment re #915

comment:48 Changed 3 years ago by skylar

In 5370//cluster/svnroot:

no longer need rsync_svn.exclude re #915

comment:49 Changed 3 years ago by skylar

In 5371//cluster/svnroot:

build squashfs re #915

comment:50 Changed 3 years ago by skylar

In 5372//cluster/svnroot:

initialize config DIRS re #915

comment:51 Changed 3 years ago by skylar

comment:52 Changed 3 years ago by skylar

In 5374//cluster/svnroot:

mkdtemp returns string not object re #915

comment:53 Changed 3 years ago by skylar

In 5375//cluster/svnroot:

mkdtemp returns string not object re #915

comment:54 Changed 3 years ago by skylar

In 5376//cluster/svnroot:

format package log re #915

comment:55 Changed 3 years ago by skylar

In 5377//cluster/svnroot:

passing logger to package install re #915

comment:56 Changed 3 years ago by skylar

In 5378//cluster/svnroot:

function to create ISO image re #915

comment:57 Changed 3 years ago by skylar

In 5379//cluster/svnroot:

packages list should be whitespace delimited re #915

comment:58 Changed 3 years ago by skylar

In 5380//cluster/svnroot:

better string cat re #915

comment:59 Changed 3 years ago by skylar

In 5381//cluster/svnroot:

null detection re #915

comment:60 Changed 3 years ago by skylar

In 5382//cluster/svnroot:

better key definedness check re #915

comment:61 Changed 3 years ago by skylar

In 5383//cluster/svnroot:

cast SVN revision to string re #915

comment:62 Changed 3 years ago by skylar

In 5384//cluster/svnroot:

better debugging re #915

comment:63 Changed 3 years ago by skylar

In 5385//cluster/svnroot:

better debugging re #915

comment:64 Changed 3 years ago by skylar

In 5386//cluster/svnroot:

paren balancing re #915

comment:65 Changed 3 years ago by skylar

In 5387//cluster/svnroot:

convert ConfigParser? object to dictionary re #915

comment:66 Changed 3 years ago by skylar

In 5388//cluster/svnroot:

do not need list of lists re #915

comment:67 Changed 3 years ago by skylar

In 5389//cluster/svnroot:

more efficient concatenation re #915

comment:68 Changed 3 years ago by skylar

In 5390//cluster/svnroot:

import ignore_patterns re #915

comment:69 Changed 3 years ago by skylar

In 5391//cluster/svnroot:

use copytree re #915

comment:70 Changed 3 years ago by skylar

In 5392//cluster/svnroot:

balance / re #915

comment:71 Changed 3 years ago by skylar

In 5393//cluster/svnroot:

adding debug statement for run_command re #915

comment:72 Changed 3 years ago by skylar

In 5394//cluster/svnroot:

mksquashfs option typo re #915

comment:73 Changed 3 years ago by skylar

In 5397//cluster/svnroot:

create ISO image re #915

comment:74 Changed 3 years ago by skylar

In 5398//cluster/svnroot:

wrong key name re #915

comment:75 Changed 3 years ago by skylar

In 5399//cluster/svnroot:

generate ISO in output directory re #915

comment:76 Changed 3 years ago by skylar

In 5400//cluster/svnroot:

use DEFUALT OUTDIR re #915

comment:77 Changed 3 years ago by skylar

In 5401//cluster/svnroot:

try to download bccd linux image re #915

comment:78 Changed 3 years ago by skylar

In 5402//cluster/svnroot:

include release in config file re #915

comment:79 Changed 3 years ago by skylar

In 5403//cluster/svnroot:

create package temp directory re #915

comment:80 Changed 3 years ago by skylar

In 5404//cluster/svnroot:

build bccd pkg re #915

comment:81 Changed 3 years ago by skylar

In 5405//cluster/svnroot:

add directory configuration re #915

comment:82 Changed 3 years ago by skylar

In 5406//cluster/svnroot:

no longer need to clear out directory dictionary re #915

comment:83 Changed 3 years ago by skylar

In 5407//cluster/svnroot:

full path for directory creation re #915

comment:84 Changed 3 years ago by skylar

comment:85 Changed 3 years ago by skylar

In 5409//cluster/svnroot:

parameter typo re #915

comment:86 Changed 3 years ago by skylar

In 5410//cluster/svnroot:

differentiate build directories from others re #915

comment:87 Changed 3 years ago by skylar

In 5411//cluster/svnroot:

initialize non-build dir dictionary re #915

comment:88 Changed 3 years ago by skylar

In 5412//cluster/svnroot:

key update re #915

comment:89 Changed 3 years ago by skylar

In 5413//cluster/svnroot:

deal with ISO directory existing before copytree re #915

comment:90 Changed 3 years ago by skylar

In 5414//cluster/svnroot:

output bccd package to build temp directory re #915

comment:91 Changed 3 years ago by skylar

comment:92 Changed 3 years ago by skylar

In 5416//cluster/svnroot:

list of directories not to create automatically re #915

comment:93 Changed 3 years ago by skylar

In 5417//cluster/svnroot:

list of directories not to create automatically re #915

comment:94 Changed 3 years ago by skylar

In 5418//cluster/svnroot:

log exclusion re #915

comment:95 Changed 3 years ago by skylar

In 5419//cluster/svnroot:

configparser reads keys as lower-case re #915

comment:96 Changed 3 years ago by skylar

In 5420//cluster/svnroot:

RELEASE is in DEFAULT section re #915

comment:97 Changed 3 years ago by skylar

In 5421//cluster/svnroot:

cast explicitly to list re #915

comment:98 Changed 3 years ago by skylar

In 5422//cluster/svnroot:

RELEASE is in DEFAULT section re #915

comment:99 Changed 3 years ago by skylar

In 5423//cluster/svnroot:

string concatenation re #915

comment:100 Changed 3 years ago by skylar

In 5424//cluster/svnroot:

paren balance re #915

comment:101 Changed 3 years ago by skylar

apparently python ConfigParser? keys method returns keys from all sections even if called on a specific section

switch to json I guess

comment:102 Changed 3 years ago by skylar

In 5426//cluster/svnroot:

switching to YAML re #915

comment:103 Changed 3 years ago by skylar

In 5427//cluster/svnroot:

match key in BUILD_DIRS re #915

comment:104 Changed 3 years ago by skylar

In 5428//cluster/svnroot:

adding DEBMIRROR re #915

comment:105 Changed 3 years ago by skylar

In 5429//cluster/svnroot:

debug setup_isolinux re #915

comment:106 Changed 3 years ago by skylar

In 5487//cluster/svnroot:

replace isc-dhcp-client due to dhclient.conf re #915

comment:107 Changed 3 years ago by skylar

In 5488//cluster/svnroot:

run fpm with replace in chroot re #915

comment:108 Changed 3 years ago by skylar

In 5489//cluster/svnroot:

coffee fail re #915

comment:109 Changed 3 years ago by skylar

In 5490//cluster/svnroot:

fpm runs in bigfe re #915

comment:110 Changed 2 years ago by skylar

In 5491//cluster/svnroot:

fix run_command re #915

comment:111 Changed 2 years ago by skylar

In 5492//cluster/svnroot:

source path should be after flags re #915

comment:112 Changed 2 years ago by skylar

In 5493//cluster/svnroot:

improved command error output re #915

comment:113 Changed 2 years ago by skylar

In 5494//cluster/svnroot:

add architecture re #915

comment:114 Changed 2 years ago by skylar

In 5495//cluster/svnroot:

use arch for args re #915

comment:115 Changed 2 years ago by skylar

In 5496//cluster/svnroot:

args type re #915

comment:116 Changed 2 years ago by skylar

  • Milestone set to 3.4.0-build_ng
  • Status changed from new to assigned

comment:117 Changed 2 years ago by skylar

In 5745//cluster/svnroot:

interim commit, moving files into src tree re #915

comment:118 Changed 2 years ago by skylar

Todo:

  • Deal with chmod/chown
  • Build modules deb for each architecture
  • deb for XFCE local files
Last edited 2 years ago by skylar (previous) (diff)

comment:119 Changed 2 years ago by skylar

In 5746//cluster/svnroot:

bccd-specific skel re #915

comment:120 Changed 2 years ago by skylar

In 5747//cluster/svnroot:

fixing paths for auxiliary scripts re #915

comment:121 Changed 2 years ago by skylar

In 5748//cluster/svnroot:

might not need ssh customizations anymore re #915

comment:122 Changed 2 years ago by skylar

In 5749//cluster/svnroot:

might not need ssh customizations anymore re #915

comment:123 Changed 2 years ago by skylar

In 5750//cluster/svnroot:

cleaning up build script re #915

comment:124 Changed 2 years ago by skylar

In 5751//cluster/svnroot:

interim commit, through line 1340 in build_livecd.pl re #915

comment:125 Changed 2 years ago by skylar

In 5752//cluster/svnroot:

no longer need trees/home re #915

comment:126 Changed 2 years ago by skylar

In 5753//cluster/svnroot:

interim commit, through line 1507 in build_livecd.pl re #915

comment:127 Changed 2 years ago by skylar

In 5754//cluster/svnroot:

final commit re #915

comment:128 Changed 2 years ago by skylar

for posterity, original build command:

bin/build_livecd.py --arch amd64 --loglevel DEBUG --nocleanup

comment:129 Changed 2 years ago by skylar

In 5755//cluster/svnroot:

interim commit re #915

comment:130 Changed 2 years ago by skylar

In 5756//cluster/svnroot:

test deb target re #915

comment:131 Changed 2 years ago by skylar

In 5757//cluster/svnroot:

use PATH re #915

comment:132 Changed 2 years ago by skylar

In 5758//cluster/svnroot:

add target directory re #915

comment:133 Changed 2 years ago by skylar

this is a test

comment:134 Changed 2 years ago by skylar

test is a 2nd test

comment:135 Changed 2 years ago by skylar

this is a 3rd test

comment:136 Changed 2 years ago by skylar

this is a 4th test

comment:137 Changed 2 years ago by skylar

In 5759//cluster/svnroot:

clean up etc re #915

comment:138 Changed 2 years ago by skylar

for next time: figure out if we can install packages via the post-install of bccd.noarch.deb

comment:139 Changed 2 years ago by skylar

In 5793//cluster/svnroot:

keys now in deb package re #915

comment:140 Changed 23 months ago by skylar

In 5803//cluster/svnroot:

debootstrap target re #915

comment:141 Changed 23 months ago by skylar

In 5804//cluster/svnroot:

need pathspec re #915

comment:142 Changed 23 months ago by skylar

In 5805//cluster/svnroot:

generate squashfs re #915

comment:143 Changed 23 months ago by skylar

In 5806//cluster/svnroot:

fix path to isolinux re #915

comment:144 Changed 23 months ago by skylar

In 5807//cluster/svnroot:

absolute path re #915

comment:145 Changed 23 months ago by skylar

In 5808//cluster/svnroot:

use iso directory re #915

comment:146 Changed 23 months ago by skylar

In 5809//cluster/svnroot:

quotes re #915

comment:147 Changed 23 months ago by skylar

In 5810//cluster/svnroot:

do not delete if failed re #915

comment:148 Changed 5 months ago by skylar

In 6001//cluster/svnroot:

set mtime of bccd-revision to bccd.noarch.deb if it is available, to avoid unnecessary deb build re #915

comment:149 Changed 5 months ago by skylar

In 6002//cluster/svnroot:

fix if statement re #915

comment:150 Changed 5 months ago by skylar

In 6003//cluster/svnroot:

use shell syntax re #915

comment:151 Changed 3 months ago by skylar

In 6023//cluster/svnroot:

unmount /proc if it is still mounted re #915

comment:152 Changed 3 months ago by skylar

In 6024//cluster/svnroot:

run /proc unmount in trap re #915

comment:153 Changed 3 months ago by skylar

In 6025//cluster/svnroot:

separate script for squashfs generation so we can unmount /proc if needed re #915

comment:154 Changed 3 months ago by skylar

In 6026//cluster/svnroot:

input validation re #915

comment:155 Changed 3 months ago by skylar

In 6027//cluster/svnroot:

c/p error re #915

comment:156 Changed 2 months ago by skylar

In 6034//cluster/svnroot:

need to unmount /sys as well re #915

comment:157 Changed 2 months ago by skylar

In 6035//cluster/svnroot:

need to unmount /sys as well re #915

comment:158 Changed 2 months ago by skylar

In 6036//cluster/svnroot:

unmount /sys in ISO build re #915, #1003

comment:159 Changed 7 weeks ago by skylar

In 6043//cluster/svnroot:

use find rather than test so we do not have a failure if bccd.noarch.deb does not exist re #915

comment:160 Changed 7 weeks ago by skylar

In 6044//cluster/svnroot:

use find rather than test so we do not have a failure if bccd.noarch.deb does not exist re #915

comment:161 Changed 11 days ago by skylar

comment:162 Changed 11 days ago by skylar

will need script to auto-generate Jenkins config.xml files from template

comment:163 Changed 11 days ago by skylar

In 6064//cluster/svnroot:

directory for jenkins config templates re #915

comment:164 Changed 11 days ago by skylar

In 6065//cluster/svnroot:

initial XML check-in re #915

comment:165 Changed 11 days ago by skylar

In 6066//cluster/svnroot:

correct name re #915

comment:166 Changed 11 days ago by skylar

In 6067//cluster/svnroot:

beginning of script to generate Jenkins job pipeline re #915

comment:167 Changed 4 days ago by skylar

In 6068//cluster/svnroot:

config files should be called config.xml re #915

comment:168 Changed 4 days ago by skylar

In 6069//cluster/svnroot:

remove extraneous BCCD re #915

comment:169 Changed 4 days ago by skylar

In 6070//cluster/svnroot:

remove STAGE and extraneous bccd from config files re #915

comment:170 Changed 4 days ago by skylar

In 6071//cluster/svnroot:

get dictionary of config data re #915

Note: See TracTickets for help on using tickets.