Stephen Smoogen: Proposed Change To EPEL Policies: Release based package lifetimes

Share Button

https://fedoraproject.org/wiki/EPEL/Changes/Release_based_package_lifetimes

Summary

The change moves expected package lifetime to that of a Red Hat Enterprise Linux minor release.

Owner

  • Name: Stephen Smoogen

Detailed Description

Extra Packages for Enterprise Linux is a sub-project of Fedora which recompiles various Fedora packages against various Red Hat Enterprise Linux releases. When EPEL was started, RHEL lifetimes were 5 years and it was thought that the repository could have similarly long lifetimes. Major rebasing of versions were not to be allowed, and fast moving software was frowned on.

Over time, the lifetime of a RHEL release grew, and the way RHEL releases rebased themselved overtime also changed. This has meant that packagers in EPEL were bound to support software longer than RHEL did and unable to rebase like RHEL could.

The current proposal is closely linked to release based composes but does not depend on it. The changes are as follows:

Packagers commit to maintaining software in EPEL for at least one RHEL minor release or 13 months, whichever is shorter. If a packager needs to stop maintaining the software, they should do the following:

  • announce on epel-devel list that they are no longer able to maintain the package and give the reasons. If someone can take it over they can do so here.
  • release one final version with an additional file named README-RETIRED.txt’ in the %docs section which says this software is retired.
  • wait until that package arrives in updates.
  • let the EPEL release manager know that the package needs to be retired for the next release.

Otherwise the latest update of the package will be retagged for the next compose of EPEL and will appear without problems.

If the situation requires a major ABI/API change (security, a new minor compose occurs, a new LTS package is released) a packager should announce to epel-devel and put in a ticket in the epel pagure instance to track. Updates can then be made and will show up in either /updates/ or the next major.minor compose.

Benefit to Fedora

  • Packagers will feel more likely to make their packages available in EPEL.
  • Users of EPEL will have more software and know it is being kept up.

Powered by WPeMatico

Share Button

Stephen Smoogen: Proposed Change to EPEL Policies: Minor Release Based Composes

Share Button


I proposed this on the EPEL list earlier this week, but realized it needed more views so am putting a version on the blog. The canonical version is at https://fedoraproject.org/wiki/EPEL/Changes/Minor_release_based_composes

Summary

The change moves EPEL composes to biannual based composes and adds an updates tree for consumers. Package trees will have a naming structure similar to Fedora release names, and will be regularly archived off to /pub/archives after the next minor release.

Package lifetimes will be similarly affected with the expected minimum ‘support’ lifetime of any package to be that of a minor release.

Owner

  • Name: Stephen Smoogen
  • Email: smooge@fedoraproject.org

Detailed Description

Extra Packages for Enterprise Linux is a sub-project of Fedora which
recompiles various Fedora packages against various Red Hat Enterprise Linux releases. Currently these packages are composed by the Fedora release engineering tools similar to Fedora Rawhide where only the latest packages tagged for a particular EPEL release (example epel-7) appear. When a package is updated or retired, the older version falls out of the EPEL repositories and users can not downgrade to older versions.

The tree structure would look like the following:


/pub/epel/releases/Major.Minor.YYYYMM/{Stuff,Modular}/{x86_64,ppc64le,aarch64,s390x,source}/{Packages,repodata}
/pub/epel/updates/Major.Minor.YYYYMM/{Stuff,Modular}/{x86_64,ppc64le,aarch64,s390x,source}/{Packages,repodata}

with 2 sets of symlinks pointing to the latest supported tree:


/pub/epel/releases/7.06.201903/Stuff/x86_64/Packages/
/pub/epel/releases/8.00.2019MM/Modular/source/Packages/

/pub/epel/updates/7.06.201903/Stuff/aarch64/Packages/
/pub/epel/updates/8.00.2019MM/Modular/s390x/Packages/

/pub/epel/releases/7 -> /pub/epel/updates/7.06.201903
/pub/epel/updates/7  -> /pub/epel/updates/7.06.201903

The proposed change will move EPEL minor releases to match with Red Hat minor releases.  For example, on 2019-02-10, Red Hat has Red Hat Enterprise Linux 7.6 in Full Support/Maintenance mode and has released a beta for RHEL-8. With the change, EPEL would compose a special set of trees for March 2019 and then all updates for those packages would go into the appropriate /updates/ sub-tree.

When the next minor release is released from Red Hat, a new compose tag will be made in koji. Packages which are not retired will then be pulled into the tag, and proven packagers can stage any mass rebuilds needed for rebases. When testing shows that this tree is ready, the symlinks will then be moved to the new tree, and the old tree will be prepared to move over to /pub/archives
Packages which are added to EPEL after a major.minor compose will only show up in the /updates/ tree until the next major.minor compose. This is the same as packages in Fedora.

Due to the fact that RHEL-6 has only a year and a half of Maintenance mode, we are not looking at making changes unless it turns out that we have to.

Benefit to Fedora

  • Users are able to downgrade to an earlier version if an update has issues for them
  • Packages will less likely break for Scientific Linux and CentOS users during minor updates.
  • Packages will be regularly archived off to /pub/archives/epel/ and consumers will less likely pull large number of old packages out of koji looking for older software.

Scope

  • Proposal Owners:
    • Add koji tags for date releases
    • Write scripts for releng to make new releases
  • Other developers:
  • Policies and guidelines:
  • Trademark approval: N/A

Known Change Impacts

Users who have hard-coded repositories or mirror specific directory changes may have a ‘broken’ experience due to symlink changes.
Archives will grow every 6 months as packages are put there regularly.

How To Test

We will stage this set of changes in /pub/alt/epel before rolling out to the main EPEL. Users wishing to test this can set up systems that will point to this tree using a temporary .repo file to be provided for testing.

User Experience

User’s should be able to do the following:

  • yum install package-name
  • yum upgrade package-name
  • yum downgrade package-name

Contingency Plan

  • Contingency mechanism: (What to do?  Who will do it?) If testing shows this proposal to fail we will not release.
  • Contingency deadline: 2019-04-01
  • Blocks release? No (we don’t do releases yet).
  • Blocks product? Yes

Release Notes

Discussed on epel-devel@lists.fedoraproject.org list with updates taken into concern.

Powered by WPeMatico

Share Button

Fedora Community Blog: FPgM report: 2019-07

Share Button

Fedora Program Manager weekly report on Fedora Project development and progress

Here’s your report of what has happened in Fedora Program Management this week.

I’ve set up weekly office hours in #fedora-meeting-1. Drop by if you have any questions or comments about the schedule, Changes, elections, or anything else.

Announcements

Meetings

Events

Fedora 30 Status

Fedora 30 includes a Change that will cause ambiguous python shebangs to error.  A list of failing builds is available on Taskotron.

Fedora 30 includes a Change that will remove glibc langpacks from the buildroot. See the devel mailing list for more information and impacted packages.

Changes

Submitted to FESCo

Approved by FESCo

Fedora 31 Status

Changes

Announced

Approved by FESCo

The post FPgM report: 2019-07 appeared first on Fedora Community Blog.

Powered by WPeMatico

Share Button

Daniel Pocock: SFK, OSCAL and Toastmasters expanding into Kosovo

Share Button

Back in August 2017, I had the privilege of being invited to support the hackathon for women in Prizren, Kosovo. One of the things that caught my attention at this event was the enthusiasm with which people from each team demonstrated their projects in five minute presentations at the end of the event.

This encouraged me to think about further steps to support them. One idea that came to mind was introducing them to the Toastmasters organization. Toastmasters is not simply about speaking, it is about developing leadership skills that can be useful for anything from promoting free software to building successful organizations.

I had a look at the Toastmasters club search to see if I could find an existing club for them to visit but there doesn’t appear to be any in Kosovo or neighbouring Albania.

Starting a Toastmasters club at the Innovation Centre Kosovo

In January, I had a conference call with some of the girls and explained the idea. They secured a venue, Innovation Centre Kosovo, for the evening of 11 February 2019.

Albiona and I met on Saturday, 9 February and called a few people we knew who would be good candidates to give prepared speeches at the first meeting. They had 48 hours to prepare their Ice Breaker talks. The Ice Breaker is a 4-6 minute talk that people give at the beginning of their Toastmasters journey.

Promoting the meeting

At our club in EPFL Lausanne, meetings are promoted on a mailing list. We didn’t have that in Kosovo but we were very lucky to be visited by Sara Koci from the morning TV show. Albiona and I were interviewed on the rooftop of the ICK on the day of the meeting.

The first meeting

That night, we had approximately 60 people attend the meeting.

Albiona acted as the meeting presider and trophy master and I was the Toastmaster. At the last minute we found volunteers for all the other roles and I gave them each an information sheet and a quick briefing before opening the meeting.

One of the speakers, Dion Deva, has agreed to share the video of his talk publicly:

The winners were Dhurata, best prepared speech, Arti, best impromptu speech and Ardora for best evaluation:

After party

Afterwards, some of us continued around the corner for pizzas and drinks and discussion about the next meeting.

Future events in Kosovo and Albania

Software Freedom Kosovo will be back from 4-7 April 2019 and I would encourage people to visit.

OSCAL in Tirana, Albania is back on 18-19 May 2019 and they are still looking for extra speakers and workshops.

Many budget airlines now service Prishtina from all around Europe – Prishtina airport connections, Tirana airport connections.

Powered by WPeMatico

Share Button

Fedora Magazine: How to watch for releases of upstream projects

Share Button

Do you want to know when a new version of your favorite project is released? Do you want to make your job as packager easier? If so, this article is for you. It introduces you to the world of release-monitoring.org. You’ll see how it can help you catch up with upstream releases.

What is release-monitoring.org?

The release-monitoring.org is a combination of two applications: Anitya and the-new-hotness.

Anitya is what you can see when visiting release-monitoring.org. You can use it to add and manage your projects. Anitya also checks for new releases periodically.

The-new-hotness is an application that catches the messages emitted by Anitya. It creates a Bugzilla issue if the project is mapped to a Fedora package.

How to use release-monitoring.org

Now that you know how it works, let’s focus on how you can use it.

Index page of release-monitoring.org

First think you need to do is to log in. Anitya provides a few options you can use to log in, including the Fedora Account System (FAS), Yahoo!, or a custom OpenID server.

Login page

When you’re logged in, you’ll see new options in the top panel.

Anitya top panel

Add a new project

Now you can add a new project. It’s always good to check whether the project is already added.

Add project form

Next, fill in the information about the project:

  • Project name – Use the upstream project name
  • Homepage – Homepage of the project
  • Backend – Backend is simply the web hosting where the project is hosted. Anitya offers many backends you can chose from. If you can’t find a backend for your project, you can use the custom backend. Every backend has its own additional fields. For example, BitBucket has you specify owner/project.
  • Version scheme – This is used to sort received versions. Right now, Anitya only supports RPM version scheme.
  • Version prefix – This is the prefix that is stripped from any received version. For example, if the tag on GitHub is version_1.2.3, you would use version_ as version prefix. The version will then be presented as 1.2.3. The version prefix v is stripped automatically.
  • Check latest release on submit – If you check this, Anitya will do an initial check on the project when submitted.
  • Distro – The distribution in which this project is used. This could be also added later.
  • Package – The project’s packaged name in the distribution. This is required when the Distro field is filled in.

When you’re happy with the project, submit it. Below you can see how your project may look after you submit.

Project page

Add a new distribution mapping

If you want to map the project to a package on a specific distribution, open up the project page first and then click on Add new distribution mapping.

Add distribution mapping form

Here you can chose any distribution already available in Anitya, fill in the package name, and submit it. The new mapping will show up on the project page.

Automatic filing of Bugzilla issues

Now you created a new project and created a mapping for it. This is nice, but how does this help you as a packager? This is where the-new-hotness comes into play.

Every time the-new-hotness sees a new update or new mapping message emitted by Anitya, it checks whether this project is mapped to a package in Fedora. For this to work, the project must have a mapping to Fedora added in Anitya.

If the package is known, the-new-hotness checks the notification setting for this package. That setting can be changed here. The last check the-new-hotness does is whether the version reported by Anitya is newer than the current version of this package in Fedora Rawhide.

If all those checks are positive, the new Bugzilla issue is filed and a Koji scratch build started. After the Koji build is finished, the Bugzilla is updated with output.

Future plans for release-monitoring.org

The release-monitoring.org system is pretty amazing, isn’t it? But this isn’t all. There are plenty of things planned for both Anitya and the-new-hotness. Here’s a short list of future plans:

Anitya

  • Add libraries.io consumer – automatically check for new releases on libraries.io, create projects in Anitya and emit messages about updates
  • Use Fedora package database to automatically guess the package name in Fedora based on the project name and backend
  • Add semantic and calendar version scheme
  • Change current cron job to service: Anitya checks for new versions periodically using a cron job. The plan is to change this to a service that checks projects using queues.
  • Support for more than one version prefix

the-new-hotness

  • File Github issues for Flathub projects when a new version comes out
  • Create pull requests in Pagure instead of filing a Bugzilla issue
  • Move to OpenShift – this should make deployment much easier than how it is now
  • Convert to Python 3 (mostly done)

Both

  • Conversion to fedora-messaging – This is already in progress and should make communication between Anitya and the-new-hotness more reliable.

Photo by Alexandre Debiève on Unsplash.

Powered by WPeMatico

Share Button

How to watch for releases of upstream projects

Share Button

Do you want to know when a new version of your favorite project is released? Do you want to make your job as packager easier? If so, this article is for you. It introduces you to the world of release-monitoring.org. You’ll see how it can help you catch up with upstream releases.

What is release-monitoring.org?

The release-monitoring.org is a combination of two applications: Anitya and the-new-hotness.

Anitya is what you can see when visiting release-monitoring.org. You can use it to add and manage your projects. Anitya also checks for new releases periodically.

The-new-hotness is an application that catches the messages emitted by Anitya. It creates a Bugzilla issue if the project is mapped to a Fedora package.

How to use release-monitoring.org

Now that you know how it works, let’s focus on how you can use it.

Index page of release-monitoring.org

First think you need to do is to log in. Anitya provides a few options you can use to log in, including the Fedora Account System (FAS), Yahoo!, or a custom OpenID server.

Login page

When you’re logged in, you’ll see new options in the top panel.

Anitya top panel

Add a new project

Now you can add a new project. It’s always good to check whether the project is already added.

Add project form

Next, fill in the information about the project:

  • Project name – Use the upstream project name
  • Homepage – Homepage of the project
  • Backend – Backend is simply the web hosting where the project is hosted. Anitya offers many backends you can chose from. If you can’t find a backend for your project, you can use the custom backend. Every backend has its own additional fields. For example, BitBucket has you specify owner/project.
  • Version scheme – This is used to sort received versions. Right now, Anitya only supports RPM version scheme.
  • Version prefix – This is the prefix that is stripped from any received version. For example, if the tag on GitHub is version_1.2.3, you would use version_ as version prefix. The version will then be presented as 1.2.3. The version prefix v is stripped automatically.
  • Check latest release on submit – If you check this, Anitya will do an initial check on the project when submitted.
  • Distro – The distribution in which this project is used. This could be also added later.
  • Package – The project’s packaged name in the distribution. This is required when the Distro field is filled in.

When you’re happy with the project, submit it. Below you can see how your project may look after you submit.

Project page

Add a new distribution mapping

If you want to map the project to a package on a specific distribution, open up the project page first and then click on Add new distribution mapping.

Add distribution mapping form

Here you can chose any distribution already available in Anitya, fill in the package name, and submit it. The new mapping will show up on the project page.

Automatic filing of Bugzilla issues

Now you created a new project and created a mapping for it. This is nice, but how does this help you as a packager? This is where the-new-hotness comes into play.

Every time the-new-hotness sees a new update or new mapping message emitted by Anitya, it checks whether this project is mapped to a package in Fedora. For this to work, the project must have a mapping to Fedora added in Anitya.

If the package is known, the-new-hotness checks the notification setting for this package. That setting can be changed here. The last check the-new-hotness does is whether the version reported by Anitya is newer than the current version of this package in Fedora Rawhide.

If all those checks are positive, the new Bugzilla issue is filed and a Koji scratch build started. After the Koji build is finished, the Bugzilla is updated with output.

Future plans for release-monitoring.org

The release-monitoring.org system is pretty amazing, isn’t it? But this isn’t all. There are plenty of things planned for both Anitya and the-new-hotness. Here’s a short list of future plans:

Anitya

  • Add libraries.io consumer – automatically check for new releases on libraries.io, create projects in Anitya and emit messages about updates
  • Use Fedora package database to automatically guess the package name in Fedora based on the project name and backend
  • Add semantic and calendar version scheme
  • Change current cron job to service: Anitya checks for new versions periodically using a cron job. The plan is to change this to a service that checks projects using queues.
  • Support for more than one version prefix

the-new-hotness

  • File Github issues for Flathub projects when a new version comes out
  • Create pull requests in Pagure instead of filing a Bugzilla issue
  • Move to OpenShift – this should make deployment much easier than how it is now
  • Convert to Python 3 (mostly done)

Both

  • Conversion to fedora-messaging – This is already in progress and should make communication between Anitya and the-new-hotness more reliable.

Photo by Alexandre Debiève on Unsplash.

Powered by WPeMatico

Share Button

Bodhi: Bodhi 3.13.0 released

Share Button

This is a feature release.

Deprecations

  • Authentication with OpenID is deprecated (#1180).
  • bodhi-untag-branched is deprecated (#1197).
  • The Update’s title field is deprecated (#1542).
  • Integration with pkgdb is deprecated (#1970).
  • Support for Python 2 is deprecated (#1871 and #2856).
  • The /masher/ view is deprecated (#2024).
  • bodhi-monitor-composes is deprecated (#2171).
  • The stacks feature is deprecated (#2241).
  • bodhi-manage-releases is deprecated (#2420).
  • Anonymous comments are deprecated (#2700).
  • The ci_url attribute on Builds is deprecated (#2782).
  • The active_releases query parameter on the Updates query URL is deprecated (#2815).
  • Support for fedmsg is deprecated (#2838).
  • Support for Bodhi 1’s URL scheme is deprecated (#2869 and #2903).
  • The /admin/ API is deprecated (#2899).
  • The fedmsg UI integration is deprecated (#2913).
  • CVE support is deprecated (#2915).

Dependency changes

  • Bodhi no longer requires iniparse (a910b61).
  • Bodhi now requires fedora_messaging (e30c5f2).

Server upgrade instructions

This release contains database migrations. To apply them, run:

$ sudo -u apache /usr/bin/alembic -c /etc/bodhi/alembic.ini upgrade head

Features

  • A new bodhi-shell CLI is included with the server package that initialized a Python shell
    with some handy aliases that are useful for debugging Bodhi (#1792).
  • Updates that obsolete security updates will now be set as security updates themselves
    (#1798)
  • The CLI no longer crashes when creating multiple buildroot overrides in one command
    (#2031).
  • The CLI can now display information about waivers (#2267).
  • Releases can now be marked as not being composed by Bodhi, which will be useful if we want to use
    Bodhi to tag builds into releases without composing those releases, e.g., Fedora Rawhide
    (#2317).
  • BuildrootOverrides are now prevented for builds that fail the test gating status (#2537).
  • The web interface now displays instructions for installing updates (#2799).
  • The CLI now has flags that allow users to override the OpenID API URL to use. This is useful
    for deployments that aren’t for Fedora and also for developers, who can use it to authenticate
    against a different OpenID server than Fedora’s production instance (Fedora’s staging instance is
    nice to use for this) (#2820).
  • The web UI search box uses a slightly longer delay before issuing the search request
    (51c2fa8).
  • Messages can now be published by fedora_messaging, a replacement for fedmsg
    (e30c5f2).
  • Associating updates with private Bugzilla tickets is now handled gracefully (7ac316a).

Bug fixes

  • The bodhi-approve-testing CLI script is now more resilient against failures (#1016).
  • The update edit page will return a HTTP 403 instead of a HTTP 400 if a user tries to edit an
    update they don’t have permissions to edit (#1737).
  • The bodhi CLI now has a --wait flag when creating BuildrootOverrides that will cause
    bodhi to wait on Koji to finish adding the override to the buildroot before exiting
    (#1807).
  • The waive button is now displayed on locked updates (#2271).
  • Editing an update with the CLI no longer sets the type back to the default of bugfix
    (#2528).
  • bodhi-approve-testing now sets up a log handler (#2698).
  • Some missing commands were added to the bodhi man page (1e6c259).
  • A formatting issue was fixed in the command line client (996b4ec).

Development improvements

  • bodhi-ci now has an integration test suite that launches a live Bodhi server, some of its
    network dependencies, and tests it with some web and CLI requests (2430433).
  • bodhi-ci‘s status report now displays the run time for finished tasks (26af5ef).
  • bodhi-ci now prints a continuous status report, which is helpful in knowing what it is
    currently doing (f3ca62a).
  • We now do type checking enforcement on bodhi-ci (2c07005).

Contributors

The following developers contributed to Bodhi 3.13.0:

  • Aurélien Bompard
  • Jeremy Cline
  • Mattia Verga
  • Ryan Lerch
  • Sebastian Wojciechowski
  • Vismay Golwala
  • Randy Barlow

Powered by WPeMatico

Share Button

Adam Young: Extract Method Refactoring in Rust

Share Button

I’m writing a simple utility for manage the /etc/hosts file. I want it in a native language so I can make it SUID, or even better, to lock it down via capabilities. I want to remember how to code in rust. Once I get a simple bit working, I want to refactor. Here’s what I did.

Here is the functioning code:

use std::env;
use std::fs;

fn words_by_line<'a>(s: &'a str) -> Vec<&'a str>> {
    s.lines().map(|line| {
        line.split_whitespace().collect()
    }).collect()
}

fn main() {
    let args: Vec = env::args().collect();
    let filename = &args[1];
    let _operation = &args[2];
    let contents = fs::read_to_string(filename)
        .expect("Something went wrong reading the file");
    
    let wbyl = words_by_line(&contents);
    for i in &wbyl {
        for j in i{
            print!("{} ", j);
        }
        println!("");
    }
}



Build and run with

cargo build
./target/debug/hostsman ./hosts list

And it spits out the contents of the local copy of /etc/hosts. We'll treat this as the unit test for now.

The next step is to start working towards a switch based on the _operation variable. To do this, I want to pull the loop that dumps the file out into its own function. And to do that, I need to figure out the type of the Vector. I use the hack of introducing an error to get the compiler to tell me the type. I change the assignment line to get:

let wbyl: u8 = words_by_line(&contents);

And that tells me:

error[E0308]: mismatched types
  --> src/main.rs:18:20
   |
18 |     let wbyl: u8 = words_by_line(&contents);
   |                    ^^^^^^^^^^^^^^^^^^^^^^^^ expected u8, found struct `std::vec::Vec`
   |
   = note: expected type `u8`
              found type `std::vec::Vec<&str>>`

So I convert the code to use that, build and run. Code now looks like this:

let wbyl: std::vec::Vec> = words_by_line(&
contents)

Now now create a function by copying the existing code block and using the variable type in the parameter list. It looks like this:

use std::env;
use std::fs;

fn words_by_line<'a>(s: &'a str) -> Vec<&'a str>> {
    s.lines().map(|line| {
        line.split_whitespace().collect()
    }).collect()
}

fn list(wbyl: std::vec::Vec<&str>>){    
    for i in &wbyl {
        for j in i{
            print!("{} ", j);
        }
        println!("");
    }
}

fn main() {
    let args: Vec = env::args().collect();
    let filename = &args[1];
    let _operation = &args[2];
    let contents = fs::read_to_string(filename)
        .expect("Something went wrong reading the file");
    
    let wbyl: std::vec::Vec<&str>> = words_by_line(&contents);

    list(wbyl);
}

Now we are prepped to continue development. Next up is to parse the command and execute a different function based on it.

Powered by WPeMatico

Share Button

Richard Hughes: PackageKit is dead, long live, well, something else

Share Button

It’s probably no surprise to many of you that PackageKit has been in maintenance mode for quite some time. Although started over ten years ago (!) it’s not really had active maintenance since about 2014. Of course, I’ve still been merging PRs and have been slinging tarballs over the wall every few months, but nothing new was happening with the project, and I’ve worked on many other things since.

I think it’s useful for a little retrospective. PackageKit was conceived as an abstraction layer over about a dozen different package management frameworks. Initially it succeeded, with a lot of front ends UIs being written for the PackageKit API, making the Linux desktop a much nicer place for many years. Over the years, most package managers have withered and died, and for the desktop at least really only two remain, .rpm and .deb. The former being handled by the dnf PackageKit backend, and the latter by aptcc.

Canonical seems to be going all in on Snaps, and I don’t personally think of .deb files as first class citizens on Ubuntu any more – which is no bad thing. Snaps and Flatpaks are better than packages for desktop software in almost every way. Fedora is concentrating on Modularity and is joining with most of the other distros with a shared Flatpak and Flathub future and seems to be thriving because of it. If course, I’m missing out a lot of other distros, but from a statistics point of view they’re unfortunately not terribly relevant. Arch users are important, but they’re also installing primarily on the command line, not using an abstraction layer or GUI. Fedora is also marching towards an immutable base image using rpmostree, containers and flatpaks, and then PackageKit isn’t only not required, but doesn’t actually get installed at all in Fedora SilverBlue.

GNOME Software and the various KDE software centers already have an abstraction in the session; which they kind of have to to support per-user flatpak applications and per-user pet containers like Fedora Toolbox. I’ve also been talking to people in the Cockpit project and they’re in the same boat, and basically agree that having a shared system API to get the installed package list isn’t actually as useful as it used to be. Of course, we’ll need to support mutable systems for a long time (RHEL!) and so something has to provide a D-Bus interface to provide that. I’m not sure whether that should be dnfdaemon providing a PackageKit-compatible API, or it should just implement a super-simple interface that’s not using an API design from the last decade. At least from a gnome-software point of view it would just be one more plugin, like we have a plugin for Flatpak, a plugin for Snap, and a plugin for PackageKit.

Comments welcome.

Powered by WPeMatico

Share Button

Richard Hughes: Using fwupd and updating firmware without using the LVFS

Share Button

The LVFS is a webservice designed to allow system OEMs and ODMs to upload firmware easily, and for it to be distributed securely to tens of millions of end users. For some people, this simply does not work for good business reasons:

  • They don’t trust me, fwupd.org, GPG, certain OEMs or the CDN we use
  • They don’t want thousands of computers on an internal network downloading all the files over and over again
  • The internal secure network has no internet connectivity

For these cases there are a few different ways to keep your hardware updated, in order of simplicity:

Download just the files you need manually

Download the .cab files you found for your hardware and then install them on the target hardware via Ansible or Puppet using fwupdmgr install foo.cab — you can use fwupdmgr get-devices to get the existing firmware versions of all hardware. If someone wants to write the patch to add JSON/XML export to fwupdmgr that would be a very welcome thing indeed.

Download and deploy firmware as part of an immutable image

If you’re shipping an image, you can just dump the .cab files into a directory in the deployment along with something like /etc/fwupd/remotes.d/immutable.conf (only on fwupd >= 1.2.3):

[fwupd Remote]
Enabled=false
Title=Vendor (Automatic)
Keyring=none
MetadataURI=file:///usr/share/fwupd/remotes.d/vendor/firmware

Then once you disable the LVFS, running fwupdmgr or fwupdtool will use only the cabinet archives you deploy in your immutable image (or even from an .rpm for that matter). Of course, you’re deploying a larger image because you might have several firmware files included, but this is how Google ChromeOS is using fwupd.

Sync all the public firmware from the LVFS to a local directory

You can use Pulp to mirror the entire contents of the LVFS (not private or embargoed firmware, for obvious reasons). Create a repo pointing to PULP_MANIFEST and then sync that on a regular basis to download the metadata and firmware. The Pulp documentation can explain how to set all this up. Make sure the local files are available from a webserver in your private network using SSL.

Then, disable the LVFS by deleting/modifying lvfs.conf and then create a myprivateserver.conf file on the clients /etc/fwupd/remotes.d:

[fwupd Remote]
Enabled=true
Type=download
Keyring=gpg
MetadataURI=https://my.private.server/mirror/firmware.xml.gz
FirmwareBaseURI=https://my.private.server/mirror

Export a share to all clients

Again, use Pulp to create a big directory holding all the firmware (currently ~10GB), and keep it synced. This time create a NFS or Samba share and export it to clients. Map the folder on clients, and then create a myprivateshare.conf file in /etc/fwupd/remotes.d:

[fwupd Remote]
Enabled=false
Title=Vendor
Keyring=none
MetadataURI=file:///mnt/myprivateshare/fwupd/remotes.d/firmware.xml.gz
FirmwareBaseURI=file:///mnt/myprivateshare/fwupd/remotes.d

Create your own LVFS instance

The LVFS is a free software Python 3 Flask application and can be set up internally, or even externally for that matter. You have to configure much more this way, including things like generating your own GPG keys, uploading your own firmware and setting up users and groups on the server. Doing all this has a few advantages, namely:

  • You can upload your own private firmware and QA it, only pushing it to stable when ready
  • You don’t ship firmware which you didn’t upload
  • You can control the staged deployment, e.g. only allowing the same update to be deployed to 1000 servers per day
  • You can see failure reports from clients, to verify if the deployment is going well
  • You can see nice graphs about how many updates are being deployed across your organisation

I’m hoping to make the satellite deployment LVFS use cases more concrete, and hopefully add some code to the LVFS to make this easier, although it’s not currently required for any Red Hat customer. Certainly a “setup wizard” would make setting up the LVFS much easier than obscure commands on the console.

Comments welcome.

Powered by WPeMatico

Share Button