Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

Eirikr1848 · 2023-05-22 07:30:07

1) Once Migration is complete -- do not notify users it is time to install new pacman version until ALL mirrors have completed their sync of new structure.
2) Only reliable way to obtain the mirror list is via the ArchLinux32 mirrorlist generator

For #2: Tried on Arch64: `reflector --latest --age` which does not always work right, apparently. Even manually installing pacman on Arch(64), then using reflector - was still prompted to downgrade pacman... even for latest 3 mirrors, sorted by age. Tried again for 1 single mirror. Same issue.

Mirrorlist migration needs to be complete, then validated, THEN announced.

The Arch team screwed up by thinking like engineers, not users.

"Git migration is complete!"
"Users must now do ____"

Without thinking of all the other issues that users may run into.

Let's be a little less hasty here on the Arch32 side and be a little more user-friendly by NOT telling our friends and family to update until all mirrors are ready for the update. This also will result in fewer headaches for us, the users, and for us, the friends/family tech support crew.

Let's do this only once.

(PS if you have pacman set to run with a cron job for them, might want to suspend that for now)

abaumann · 2023-05-22 17:34:00

To be fair: the git repo move was announced internally for quite a while and a test infrastructure was available upstream. We just didn't have time to do much..
The git repo move was overdue for quite a while, as those huge git repos were a nightmare to keep up to date on build machines.
The extra/community merge was equally overdue, it was basically a historical left-over with no real life meaning anymore.

I just upgraded two 64-bit machines without any hickups.. cannot judge the new pkgctl (yet), so users should not be too affected.

And I don't worry about the user basis of Arch32 too much (which is tiny compared to the one from upstream). :-)

reflector is not really a core component of Arch (and I think adaption has to be done by the author of reflector). There are simply
too many 3rdParty tools floating around for Arch and it's the responsability of the respective authors to fix them (they
might also gladly accept patches).

Mirror migration was not a big issue, either you had a extra/community thing or just a extra thing. community is
still available as empty stub. Also during normal operation you can have stale mirrors or similar hickups..

Arch is by engineers for engineers, so, an engineer is supposed to be able to handle those issues..

OTOH: there might be a problem for all downstream, user-friendly distros which were caught on the wrong foot by
this change. But it's too early to tell. IMHO the worst which can happen is that some distros don't have updates for
a while (and the maintainers have to adapt their build systems).

Running pacman in a cron (*urgh*), might go as well as doing Windows updates automatically (speaking as sysadmin here). :-)

Eirikr1848 · 2023-05-23 15:20:30

I saw a lot of confusion and chaos on Reddit, probably because a lot of users are new (bought a Steam Deck - want to try Arch on their main rig, etc).

So in my head it was a bigger "ordeal" than it actually was, that's on me for overreacting given that I now encounter a lot of gamers in the wild using Arch and have been essentially playing "Arch sensei" in various Discord groups unwillingly. It also seems to have become an OS for tinkerers.

You are right though, 32 should not be a problem.

(And pacman in a cron is for grandpa, who uses his dual Pentium III PC... still... and understands how cron works. He moved himself to Arch, then A32 from Debian a long time back. He forgets to update, then things break, so he set up his pacman cron to avoid outdated keys, etc. Deffo not advocating for this widespread -- was thinking very micro-scale for "that one person" with a 80-something family member still trying to keep himself sharp).

Side Note: He upgraded to a Quadro4 750XGL, dual Pentium III Xeon 900MHz/2MB L2 cache CPUs, 4GB RAM, Radeon 9700 Pro, and then a RAID card and set up his SSDs in a RAID. At this point, he wants things to simply work. So had him + Reddit comments in mind when posting. I'll tell him not to panic, and just wait 24 hours after the all-clear from the ArchLinux 32 page.)

Last edited by Eirikr1848 (2023-05-23 16:40:44)

abaumann · 2023-05-23 17:21:55

Thanks to deep42thought adaptions are ready, so the builds should resume shortly for Archlinux32. There is a HUGE backlog now and we have to rebuild ALL packages.
So I'll see if I can add some more build machines to the problem..

Steam and Arch is a problem per se and should never have been done like this (I installed "Among Us with Other Roles Mod" lately and oh boy - what a mess!).
You need 2 engineers (and one bulb) to sort out the problems.. ;-)

Eirikr1848 · 2023-05-24 10:32:41

abaumann wrote:

Thanks to deep42thought adaptions are ready, so the builds should resume shortly for Archlinux32. There is a HUGE backlog now and we have to rebuild ALL packages.
So I'll see if I can add some more build machines to the problem..
Steam and Arch is a problem per se and should never have been done like this (I installed "Among Us with Other Roles Mod" lately and oh boy - what a mess!).
You need 2 engineers (and one bulb) to sort out the problems.. ;-)

Oh no! I'm so sorry I did not see this sooner. I also have a Ryzen 5800X speedcapped a 3.4GHz, a laptop with 6x core i7-9000 series, a 5600G, and a 9900K desktop CPU available if you need some build power.
Let me know if there is a way to get something quickly (relatively so) configured for you. Everything has 32GB RAM.

I have a Ubiquiti UDM Pro with a 4-port Ubiquti hub on remote-guest VLAN for tunneling as-needed and can quickly set up a containerized/virtualized instance of your choosing for you to SSH in to. I don't want to access that environment, I don't want you to see mine. I can retool my existing tinkerlab hardware for whatever.

(Also have some lowwwww end machines. Multiple PIII Xeons w/2GB RAM, Pentium M 2.26GHz AOPen "NUCs before the NUC" w/2GB RAM, Bobcat E300 and E350 boards with 16GB RAM, Intel Atom 1st gen NUCs w/16GB RAM, Haswell NUCs with 16GB VRAM, etc.

... also what do you mean by Steam and Arch? Are... is... will... are you hinting that Steam will work on ArchLinux32?! *giggles with childish glee*

Last edited by Eirikr1848 (2023-05-24 10:35:09)

abaumann · 2023-05-24 15:57:08

That would indeed be great :-)

There is a guide in the wiki (how to set up a slave), basically then we just need the ssh key of the worker and add it to the buildmaster.
I also have to bring my eurobuild6 back to live - so far I was not successful to build a single package (I had some weird issues).

@deep42thought: is it enough to update the 'builder', I didn't see an adaption for 'devtools32' which make me wonder.. :-)

abaumann · 2023-05-24 16:15:57

Even two guides:
- https://bbs.archlinux32.org/viewtopic.php?id=2839
- https://bbs.archlinux32.org/viewtopic.php?id=5

P.S. my build slave just checks out something, then says error.. mmh.

abaumann · 2023-05-24 16:20:12

lto-wrapper: fatal error: write: No space left on device
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make[3]: *** [/build/gcc/src/gcc/gcc/c/Make-lang.in:87: cc1] Error 1
make[3]: Leaving directory '/build/gcc/src/gcc-build/gcc'
make[2]: *** [Makefile:5013: all-stage2-gcc] Error 2
make[2]: Leaving directory '/build/gcc/src/gcc-build'
make[1]: *** [Makefile:30942: stage2-bubble] Error 2
make[1]: Leaving directory '/build/gcc/src/gcc-build'
make: *** [Makefile:31154: bootstrap] Error 2
==> ERROR: A failure occurred in build().
Aborting...
==> ERROR: Build failed, check /var/lib/archbuild/staging-with-build-support-pentium4/erich/build
used straw: :with_build_support:clean_chroot:
erich@va-x8664

abaumann · 2023-05-24 16:21:00

My build logs of eurobuidl6-1 don't appear on the buildmaster, so something else must be wrong..

abaumann · 2023-05-24 16:25:35

Maybe deleting all /var/lib/archbuild and /var/cache/archbuild is a good idea..

abaumann · 2023-05-24 16:48:10

Mmh. Creating chroots failed for me, so I definitely don't have the correct devtools..

Eirikr1848 · 2023-05-25 11:14:44

OH NO! Is there a way to reach you quicker? I only check messages on forums occasionally, social media even less. Telegram? Discord? IRC?

Shouting into the void is so frustrating! I'm so sorry... lmk what I can do to help...

abaumann · 2023-05-27 08:11:49

Aeh, there is Libera Chat on #archlinux32. There shouting into the void might be less of a problem. :-)
I currently still have unexplainable errors when starting build machines, no clue what's wrong..

Eirikr1848 · 2024-01-30 02:01:03

abaumann wrote:

Aeh, there is Libera Chat on #archlinux32. There shouting into the void might be less of a problem. :-)
I currently still have unexplainable errors when starting build machines, no clue what's wrong..

Oh my gosh I never was subscribed and never saw your reply! I usually like ALWAYS subscribe to these! NOOO I feel so bad. What did the random build errors end up being!? I guess I will join libera chat sometime in that case, dang!

Arch Linux

#1 2023-05-22 07:30:07

Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#2 2023-05-22 17:34:00

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#3 2023-05-23 15:20:30

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#4 2023-05-23 17:21:55

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#5 2023-05-24 10:32:41

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#6 2023-05-24 15:57:08

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#7 2023-05-24 16:15:57

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#8 2023-05-24 16:20:12

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#9 2023-05-24 16:21:00

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#10 2023-05-24 16:25:35

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#11 2023-05-24 16:48:10

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#12 2023-05-25 11:14:44

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#13 2023-05-27 08:11:49

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

#14 2024-01-30 02:01:03

Re: Arch's Git Migration is a Mess: What Can we Learn for ArchLinux32?

Board footer