Charles-Antoine Couret: Revue de presse de Fedora 29

Share Button

Cela fait depuis Fedora 19 que je publie sur la liste de diffusion de
Fedora-fr une revue de presse de chaque sortie d’une nouvelle version.
Récapituler quels sites en parle et comment. Je le fais toujours deux semaines
après la publication (pour que tout le monde ait le temps d’en parler).
Maintenant, place à Fedora 29 !

Bien entendu je passe sous silence mon blog et le forum de fedora-fr.

Sites web d’actualité

Soit 6 sites sur les 25 contactés.

Blogs, sites persos ou sites non contactés

Soit 1 site.


Le nombre de sites parlant de Fedora 29 est en légère baisse encore.
Beaucoup d’articles se fondent sur ce que j’ai moi même rédigé (que ce soit la
version courte ou longue).

La semaine de sa sortie, nous avons eu une augmentation de visites par
rapport à la semaine d’avant de cet ordre là :

  • Forums : stable
  • Documentation : baisse de 2% (soit environ 163 visites en moins)
  • Le site Fedora-fr : 53% (soit 550 visites en plus)
  • Borsalinux-fr : 479% (soit 90 visites en plus)

À tenir compte de la situation particulière avec une sortie lors des
vacances de la Toussaint en Europe avec en plus un pont cette semaine-ci.
D’ailleurs le forum et même d’autres sections du site ont connu une hausse de
fréquentation la semaine suivant la sortie.

Si vous avez connaissance d’un autre lien, n’hésitez pas à partager !
Rendez-vous pour Fedora 30.

Powered by WPeMatico

Share Button

Fedora Community Blog: Linux Day 2018 – Italy

Share Button

Picture of Fedora stickers on a table

Every year, on the last Saturday of October, in Italy there is a national event called “Linux Day”. This year was the 18th edition and it was held on October 27.

The event is promoted by the Italian Linux Society, and it is independently organized in many cities all around the country by groups of volunteers, LUGs and various associations. Even if it is highly fragmented (many little events in many cities), it is probably the biggest Italian event related to Linux and FLOSS, that is directly organized by people involved in the communities and by ordinary users.

The aim of such event is to to promote Linux and FLOSS in general: in each city there are many talks, presentations and installation parties. The target audience is not limited to computer enthusiasts, hackers or IT professionals, but newbies, students and curious citizens are welcome as well.

Fedora representation

As far as we know, this year Fedora was represented by ambassadors and advocates in five events: in Massa Carrara (event organized by L.U.Ma.Ca.) with a talk about Silverblue oriented to ordinary Linux users held by alciregi; in Ivrea (event organized by IvLug) with a talk titled “Fedora: an up to date distro for everyone” held by dariolesca; and in Mezzago (event organized by K12 APS, BrigX Bernareggio and CoderDojoBrianza) with a talk titled “The three souls of the web in the service of the communities: information, collaboration, innovation” held by mailga. In Rieti (event organized by lugrieti) and in Cagliari (event organized by GULCh) there were Fedora booths thanks to gabrycaos and veon.

Some details from Massa Carrara (by alciregi)

In Carrara there were at least 30 people witnessing the talks, plus a lot of people jumped in just to ask something or to see some projects exposed in the dedicated area, and just to grab some swag. The attendees were mainly long time Linux users using various distributions, and some high school students. The topic of the talk related to Fedora Silverblue has produced a lot of excitement, mainly because on the Linux desktop ecosystem there have been not too many novelties in the last times. On the other side the new kind of Linux distribution was seen as too destabilizing in the classical day to day average user workflow. However, the container technology brought to the desktop was seen as an interesting opportunity, and this is particularly true for students looking to learn modern technologies for a future job in the IT field. Here you can find the slides (in Italian).

Alessio Ciregia (alciregi) presenting Fedora Silverblue, Linux Day ’18 in Massa Carrara

Some details from Ivrea (by dariolesca).

In Ivrea, in the University room “Officina H”, at least 100 people have took part to the event: someone attending to all the talks and some other jumped in just to lurk and see what kind of event it was.
There were many talks about many topics, attendees seemed very interested and compelled, the Q&A session at the end of each talk was very active.

There was a gadget booth, where people was able to grab some swag provided by Italian Linux Society and by the Fedora Project.
The PC used to show the slides of each talk was running Fedora 29 (still beta at the time).

The talk presented by me was titled “Fedora: una distro per tutti al passo coi tempi” (more or less translated to “Fedora: an up to date distro for everyone”). In addition it was showed how to keep many desktop environment on the same Fedora installation, and that it is possible to select one of them at each login. A note: Fedora was installed on an USB 3 stick.
The attendees have shown a strong interest, and many of them asked how to try it on their PC.

Here you can find the slides (in Italian).

Dario Lesca presenting to a room
Dario Lesca (dariolesca) talking about Fedora, Linux Day ’18 in Ivrea

Some details from Mezzago (by mailga)

The Linux Day held in Mezzago (“biblioteca”) didn’t saw the attendance of lots of people, maybe 20 ’cause, as usual here in Italy, the Linux Day is very fragmented. Nevertheless the talks were very interesting and the people followed all of them paying lots of attention. Among the speakers there was the presence of Italo Vignoli and this raised the level of the talks.
About my speech, I explained (mainly referred to the Fedora community) how a community is built, the dynamics within , and a sort of “netiquette” to get requested things done.

It was a half-an-hour talk so I could not dive in the deep of the communities flows. For the people who want give a glance to the talks, here the slides and the full speeches (in Italian).

Gabriele Trombini (mailga) talking about FLOSS Communities with an eye to Fedora, Linux Day 2018 in Mezzago

What we have learned

We know that the Italian Fedora Community as a whole is a little bit inactive, compared to some years ago. We know many Italian Fedora users, but there is a lack of people with a compelling interest in animating or actively participating in a community. However there are various Italian contributors, but each of us does their work under the hood as an individual, in some cases collaborating directly in the international community.

Users sometimes don’t care about the possibility to contribute to the project (to any project, actually). It seems that people look at a Linux distribution as a product, like a smartphone or a commercial software, and it is particularly true for young people: if it works it is nice, but if they encounter a problem it is more simple to complain or to switch to another distro. At a first glance, it seems difficult to on-board them as contributors. Many people is not aware that there are many ways to contribute inside a community.

What we can do in the future

For sure in the near future we can work in order to convince current contributors to head up and participate to events with talks ad workshops.
Looking to the next year Linux Day, during the coming 12 months, we could try to solicit more people (be they regular users or contributors) to submit speeches talking about the Fedora Project and to organize Fedora booths in more cities than this year event. In addition we should fight laziness, and we could try to invite regular users to contribute and look with interest to the community behind the project, and not only to the technology.

The post Linux Day 2018 – Italy appeared first on Fedora Community Blog.

Powered by WPeMatico

Share Button

Fedora Community Blog: Elections nominations now open

Share Button

Fedora 26 Supplementary Wallpapers: Vote now!

Today we are starting the Nomination & Campaign period during which
we accept nominations to the “steering bodies” of the following teams:

This period is open until 2018-Nov-28 at 23:59:59 UTC.

Candidates may self-nominate. If you nominate someone else, please check with them to ensure that they are willing to be nominated before submitting their name.

Nominees can start preparing their answers for questions in the Election Questionnaire. The questions can be found in the template files on Pagure.

Nominees submit their questionnaire answers via a private Pagure issue.  The Election Wrangler or their backup will publish the interviews to the Community Blog a day before the start of the Voting period (2018-Dec-06).

note that the interview is mandatory for all nominees. Nominees not
having their interview ready by end of the Interview period (2018-Dec-05) will be disqualified and removed from the election. Nominees may submit their interview answers immediately and may edit them until the end of the interview period.

Nominees are encouraged to begin their interview answers as soon as they accept their nomination.

As part of the Campaign people might also ask questions to specific
candidates on the devel mailing list, if they want.

The full schedule of the December 2018 Elections is available on the
Elections schedule.

The post Elections nominations now open appeared first on Fedora Community Blog.

Powered by WPeMatico

Share Button

Hans de Goede: New plymouth theme for flickerfree boot

Share Button

As discussed in my previous blog post one of my TODO list items for plymouth is creating a new plymouth theme.

Since the transition to plymouth is not entirely smooth plymouth by default will wait 5 seconds (counted from starting the kernel) before showing itself so that on systems which boot under 5 seconds it never shows. As can be seen in this video, this leads to a very non-smooth experience when the boot takes say 7 seconds as plymouth then only shows briefly, leading to a kinda “flash” effect while it briefly shows.

Another problem with the 5 second wait, is now that we do not show GRUB the user is looking at the firmware’s bootsplash for not only the often long firmware initialization time, but also for the 5 seconds plymouth waits on top, making it look as if nothing is happening.

To fix this I’ve been working on a new plymouth theme which draws a spinner over the firmware boot splash, eliminating the ugly transition from the firmware boot splash to plymouth. This also allows removing the show-delay, so that we provide feedback that something is happening as soon as plymouth starts.

Firmware being firmware getting this done right was somewhat harder then I expected, but I’ve a first “draft” of a new theme doing this now. I’ve created some videos showing 2 different systems booting the new theme:

Note the videos with diskcrypt where paused when I entered my passphrase. So there is a bit of a jump in them because of this.

I’ve built a test version of plymouth for Fedora 29, to give this a try download all rpm files from here except the .src.rpm and -devel files and then from a directory with all those files in it, run:

sudo rpm -Uvh plymouth*.rpm

Since plymouth is part of your initrd, you also need to regenerate your initrd:

sudo dracut -f /boot/initramfs-$(uname -r).img $(uname -r)

This regenerates the initrd for the kernel you are currently running, so if you’ve installed a kernel update and have not rebooted since then you may not get the new theme when rebooting. In this case rerun the dracut command after rebooting.

Now reboot and you should get the new spinner on firmware-boot-splash theme, with Fedora branding.

If you give this a try and the new theme somehow does not look correct, please mail at If you mail me about the theme not displaying correctly please attach the /run/plymouth.log file which this test-build generates to the email and a video of how the theme misbehaves would be great too.

I still need to discuss the idea of using a new theme incorporating the firmware boot splash with the GNOME design team so this is all subject to change.

Powered by WPeMatico

Share Button

Fedora Magazine: VS Code Live Share plugin

Share Button

Contributing to Open Source projects leads to collaborating with people around the world this is traditionally done via emails or instant messages. But with the rise of extreme programing practices like pair programing being able to remotely share a code editor is a great feature. VS Code has a plugin Live Share that does just that.

Getting Started with Live Share

If you do not have VS Code already installed, you can read our previous article to get started.

Using Visual Studio Code on Fedora

Installing the Live Share plugin

First let’s install the dependencies the Live Share plugin requires.

$ sudo dnf install openssl-libs krb5-libs libicu zlib gnome-keyring libsecret desktop-file-utils xorg-x11-utils

Once the installation completes, the Live Share plugin is ready to be installed from the Extensions marketplace.

More details on the role of each dependency can be found in the Documentation.

Finally reload VS Code to activate the new plugin.

Start a Collaboration Session

To start a new collaboration session, it requires a project to be open in VS Code, so let’s clone the following git repository and open it.

$ git clone

This repository contains the source code of the following Magazine article.

Never miss a Magazine article — build your own RSS notification system

In VS Code, use the [Ctrl+K, Ctrl+O] combination to select and open the git repository.

Then from the Live Share extension menu start a collaboration session.

Live Share requires its user to sign in so that it can display the identity of the session participants. You can easily sign in using a GitHub account for example.

Ready to collaborate

To start collaborating Live Share provides a link that can be shared with others. You can get this link in you clipboard by clicking on the Invite paticipants … text.

Once one or more participant have joined the session you can start collaborating.

Sharing Code

This is where Live Share really shines as it shows clearly which participant is currently editing the file.

It is also possible to follow a participant. This has for effect to open in your editor all the files that the participant you are following opens.

Sharing Resources

Another feature of Live Share is to share resources like a server, a terminal or even a debugger session. For example it is very easy to share a terminal session and help a participant setting up a development environment, another example would be to share a database server and to inspect the content of a table.


By default the Live Share connection automatically checks if the host machine and the guest machine can communicate directly, if not Live Share uses a relay hosted in the Azure cloud.

It is possible to configure Live Share to use only direct connection between the host and the guest. For that the host machine needs to open a port in the 5990-5999 range and accept inbound local network connections. The guest needs a network route and outbound access to the host on the same port.

Finally set the connection mode to direct in the Live Share settings.

The documentation provides more details on the connectivity options.

Please note that the Live Share extension is available as Public Preview. The extension source code is currently not open source but this should change once the extension is generally available.

Powered by WPeMatico

Share Button

Luya Tshimbalanga: Using AMD RX Vega driver OpenCL on Fedora 29

Share Button

The Raven Ridge APU is very capable processor to handle OpenCL inside some applications like Blender, Darktable and Gimp. Unfortunately, the current implementation from Mesa, clover, stuck to 1.3, is not supported. AMD released their driver 18.40 with OpenCL2.0+ targeting only Red Hat Enterprise Linux/Cent OS 6.10 and 7.5 in addition of Ubuntu LTS. The good new is the former rpm format can be used on Fedora.

The graphical part of Raven Ridge is Vega 8, basically a cut-down of Vega56 or Vega64 meaning choosing either driver for RX Vega.
The instruction is provided for extracting the rpm files but here is
 some requirements for OpenCL:

  • kernel-devel (provided by Fedora repository)
  • amdgpu-dkms
  • dkms
  • libopencl-amdgpu-pro
  • opencl-amdgpu-pro-icd

Once done, applications needing OpenCL will automatically detect the driver located on /opt/amdgpu/lib64. Blender will list as unknown AMD GPU and Darktable will enable it.

OpenCL from official AMD driver enabled on Darktable

Raven Ridge Vega8 listed as unknown AMD GPU detected

There is a ROCm version but it currently does not support the graphical side of Raven Ridge at this time. It will be great that someone will finally write a srpm for Fedora.

Powered by WPeMatico

Share Button

Ismael Olea: Degree final work about ISO/IEC 29110

Share Button

Cover of «Creation of artifacts for adoption of ISO/IEC 29110 standards» blueprint

I want a lot to write more in this blog. There are matters I didn’t talk enough about SuperSEC or GUADEC conferences, some announce for 2019 and some some activities in Wikipedia (specially in the Wikiproyecto-Almería and my firsts step in the amazing world of SPARQL), less important but I really enjoy.

But now I want to keep record of significant advances in the university degree I’m finishing these months. I decided to finish a pending course with special interest in the required degree final work, to work in things I’ve been interested since 2003 but never had the oportunity to focus in deep enough to study, learn and write some useful, I hope, tools. And it’s being fun 🙂

29110 Galore at

So now I can say the project blueprint has been approved by the university. It’s named «Creation of artifacts for adoption of ISO/IEC 29110 standards» (document in Spanish, sorry) and the goals are to produce a set of opensource artifacts for the adoption of the 29110 family of standards focused on a light software engineering methodology suitable to be adopted by very small entities (VSEs). At the moment my main target is to work in the «Part 5-4: Agile software development guidelines», currently on development by WG24, using the EPF Composer tool.

As a working tool I’m making a (half backed and maybe temporal) website to keep record of related materials at

Hope to announce related news in the next weeks.

Powered by WPeMatico

Share Button

Richard Hughes: Adding an optional install duration to LVFS firmware

Share Button

We’ve just added an optional feature to fwupd and the LVFS that some people might find useful: The firmware update process can now tell the user how long in seconds the update is going to take.

This means that users can know that a dock update might take 5 minutes, and so they start the update process before they go to lunch. A UEFI update will require multiple reboots and will take 45 minutes to apply, and so the user will only apply the update at the end of the day rather than losing access to the their computer for nearly an hour.

If you want to use this feature there are currently three ways to assign the duration to the update:

  • Changing the value on the LVFS admin console — the component update panel now has an extra input field to enter the
    duration in
  • Adding a new attribute to the element, for instance:
  • Adding a ‘quirk’ to fwupd, for instance:
    InstallDuration = 40
  • For updates requiring a reboot the install duration should include the time to POST the system both before and after the update has run, but it can be approximate. Only users running very new versions of fwupd and gnome-software will be shown the install duration, and older versions will be unchanged as the new property will just be ignored. It’s therefore safe to include in all versions of firmware without adding a the dependency on a specific fwupd version.

Powered by WPeMatico

Share Button

Justin W. Flory: Sustain OSS 2018: quick rewind

Share Button

Sustain OSS 2018: quick rewind

This year, I attended the second edition of the Sustain Open Source Summit (a.k.a. Sustain OSS) on October 25th, 2018 in London. Sustain OSS is a one-day discussion on various topics about sustainability in open source ecosystems. It’s also a collection of diverse roles across the world of open source. From small project maintainers to open source program managers at the largest tech companies in the world, designers to government employees, there is a mix of backgrounds in the room. Yet there is a shared context around the most systemic problems faced by open source projects, communities, and people around the world.

The shared context is the most valuable piece of the conference. As a first-time attendee, I was blown away by the depth and range of topics covered by attendees. This blog post covers a narrow perspective of Sustain OSS through the sessions I participated and co-facilitated in.

Speed breakout groups

The morning started with speed breakout groups of between six to twelve people. Several attendees acted as facilitators for discussion on special topics. Every attendee could about half of all groups. I took extensive notes in the following groups:

  • Charitable participation in open source
  • Diversity and inclusion
  • Turning open source projects into sustainable projects / companies
  • Design in open source
  • Open source financial sustainability models

Sustain OSS: High-level takeaways

To save you time, these are my high-level takeaways across all breakout groups I participated in:

  • Open source isn’t something just done in people’s free time
  • Complex systems can enable systemic bias in terms of what “open source” means
  • Sustainability as topic of first priority / consideration, not an afterthought
  • There is no “silver bullet” solution to any of these challenges; they all require adaption to work across communities, projects, and organizations

Charitable participation in open source

This breakout group focused on the connection between charitable organizations and free software projects. It was facilitated by the esteemed Karen Sandler of the Software Freedom Conservancy.

Overall, the conversation was split among creating ethical software, finding sustainable funding models, and balancing how much control to relinquish as a managing organization of an open source project. Some felt pride and ideology were strong drivers for contributors to ideological projects (which also mirrors my experience at UNICEF). These could be key motivations to understand for contributors. Additionally, the challenge around sustainable funding models was common across charitable foundations focused on free software. Grant funding is a common strategy employed by charitable organizations, but the short-term nature of grants puts additional strain on resources to continue searching for new funding. Lastly, for charitable organizations overseeing or supporting free software projects, there was uncertainty over how much control should be left to projects. Attendees generally expressed a desire to let projects do what they want, but it sometimes came at the risk of additional overhead for the organization when everyone does something of everything. The concern over toxic communities came up, and how some issues remain buried until farther along in a relationship with a project. One successful solution employed was to hold monthly meetings among all member projects of an organization to address difficulties.

One interesting detail that captured my attention: one attendee noted how extensive effort into fundraising campaigns targeted to members of a foundation actually increased member engagement with the foundation.

Diversity and inclusion

My biggest takeaway from this session was the danger in thinking of open source as something we do in our free time. This can be exclusive to different genders, races, and socioeconomic statuses. Some “free time” is more equal than others. The actionable piece for me is to be more conscious in building and growing communities to support different levels of contribution in a community.

The question I wanted to explore after reflecting is to ask of those who feel disadvantaged:

  • What factors makes a project more or less inviting for you?
  • What can we do better when designing for participation in our communities?

Turning open source projects into sustainable ones

My notes weren’t thorough on this session, but there was an interesting point on trademark that came up during discussion of the Commons Clause. One participant was pursuing trademark law to enforce commercial protections and sustainability. They gave an example of a large corporation advertising support with a major open source project (e.g. a major software/hardware vendor supporting a specific NodeJS version). They wanted to use this as a way to create a more financially sustainable model for some projects.

Design in open source

This breakout group focused on sustainable design and design practices in open source communities. The role of designers in technical projects was also discussed and how we can build technical communities to be more inclusive for designers. It was facilitated by Elio Qoshi.

My takeaways from this breakout were that established ways of working can be unfriendly to designers and there is a need to emphasize diversity across different roles in a project or organization. Certain tools, platforms, or other mechanisms for contributing have poor user interfaces. They can push people away because of barriers to contributing with a frustrating user experience. Next, the need for diversity in roles was noted, with an example of engineers leading project management. Sometimes bias or oversights afforded as an engineer accidentally excludes others like designers or writers from contributing to our project. We should endeavor for people to spend more time on their preferred and most effective methods of contribution.

Financial sustainability models

This breakout session focused on the traditional sense of sustainability: in finances and resources. Attendees discussed different models used to fund open source projects and foundations. The session was facilitated by the founder of the MusicBrainz project, Robert Kaye.

The model used by MetaBrainz essentially as a data broker was interesting and unique. MetaBrainz offers commercial data usage at a cost, and companies using their data have a strong need for the data and see value in it. Through other parts of their model since changing three years ago, they had significant gains in their revenue and were able to increase paid staff working on the projects.

The Amazon invoice cake is also an amusing story, but you should ask Robert directly about it.

Metabrainz charity sends Amazon a cake celebrating 3d anniversary of unpaid invoice.

— Boing Boing (@BoingBoing) December 5, 2013

Hour breakout sessions

After lunch, attendees participated in two hour-long breakout sessions to explore specific topics in greater detail.

Human aspect of governance

Longer form notes are available below. I won’t go into detail since it has its own document with notes and highlights.

University engagement

Together with Josh Greenberg of the Alfred P. Sloan Foundation, we co-facilitated a spontaneous session on how universities can engage with open source communities and vice versa.

In our session, two major topics were discussed:

  • Education (e.g. curriculum, institutions, programs, etc.)
  • Research

We asked all participants why they decided to participate and what questions they had, even though we weren’t able to answer all of them:

  1. How do we get the word out?
  2. What research is most valuable for open source?
  3. How to long-term sustain projects?
  4. How to actually do and support research?
  5. How to engage both students and faculty?
  6. How to harness / enable institutions to make positive contributions to ecosystem?

For education, we agreed that introducing and teaching open source in curriculum better serves students and the institution (both financially and in career satisfaction). Many technology companies today are participating in open source and it is an important skill to have for students entering the workforce. For research, students are already doing research and proposing topics, so better student engagement in open source is better for research.

Our takeaways were to better engage with existing organizations working on these problems for years already (e.g. POSSE), shifting the perspective of universities to be stewards of FOSS, and using collegiate hackathons as a way to better engage with undergraduate students.

One additional point that stood out to me was the emphasis across all breakout participants for a need of good communication skills to be successful. In many cases, the companies hiring top tech talent (from our breakout attendees) listed this as most desirable skill. Technology and new skills can be learned, but teaching good communication skills and how to work collaboratively are not easily learned.

Other takeaways

One takeaway I couldn’t fit elsewhere was my changed perspective on “technical” vs. “non-technical” work. The phrase “non-technical work” implies an “other space where development does not occur”. Does the phrase place unequal priority on technical work? One action item is to avoid using “non-technical work” as an umbrella term, and instead call these areas by what they are: design, documentation, writing, marketing, community building, etc.

For me, I still want an umbrella term for these things, but I’m open-minded for better alternatives to non-technical.

Skill share: conflict resolution

The last event of Sustain OSS was a 1×1 skill share. Roughly half of the attendees identified a “skill” they could teach someone else in the room. The other half of attendees paired with someone teaching a skill they wanted to learn more about. I paired with Jono Bacon on a short breakout on conflict resolution.

Jono detailed steps of working through and resolving conflict, including how to identify root problems, how to make steps to resolve them, and some personal philosophy of how we build and maintain relationships with others.

An important first step is to identify the critical point: this could be an ongoing crisis, dealing with interpersonal conflict, or dealing with burnout. When someone is explaining a problem, listen fully to them and understand what they are saying. Let them get it off their chest. Is there something else causing this behavior? Tap into the cloud of ranting and determine what the root cause is.

Once common ground is established, make a plan to resolve it. Jono’s advice was to create written next steps and be explicit about expectations. This way, everyone is on the same page of what the next steps are and everyone involved has signed off on these next steps (this creates a sense of commitment and the next steps become written as “law”). Encourage others to restate the goals of conflict resolution in their own words. Once you have written goals and expectations, the crucial next step is follow-up. Check in on a regular basis with the person or people involved. Try to be neutral and unbiased when listening to others in these conversations. Go in with an open mind.

Lastly, we contextualized conflict resolution in personal philosophy of how we build and maintain relationships with others – both in and out of our open source projects. Sometimes the best way to address difficult interpersonal problems is to stop avoiding them and simply address them. Much easier said than done, but otherwise there is no escaping the perpetuated cycle of conflict if someone doesn’t make a first step.

It’s not just about code.

Thank you

To wrap up this Sustain OSS report, a few obligatory thank-yous are needed:

  • Sloan Foundation / Ford Foundation: For the financial support I needed to attend and participate in the event – this is never something I take for granted and I am happy to have received a scholarship to attend and participate
  • Josh Greenberg @ Sloan Foundation: For helping me get over some imposter syndrome and co-facilitate the university engagement breakout session with me – thanks for the gentle push
  • Robert Kaye @ MetaBrainz: For being generally awesome and finally giving me someone to nerd out about all these crazy ideas of how free culture and music can actually be related!
  • Stephen Jacobs: For always being supportive for yet another trip abroad and helping me map a strategy to get the most out of Sustain OSS

Sustain OSS gave me a lot to think about and consider. I’m glad and fortunate to have attended. I hope this event report gives additional visibility to some of the conversations held in London this year.

The post Sustain OSS 2018: quick rewind appeared first on Justin W. Flory’s Blog.

Powered by WPeMatico

Share Button

Richard Hughes: More fun with libxmlb

Share Button

A few days ago I cut the 0.1.4 release of libxmlb, which is significant because it includes the last three features I needed in gnome-software to achieve the same search results as appstream-glib.

The first is something most users of database libraries will be familiar with: Bound variables. The idea is you prepare a query which is parsed into opcodes, and then at a later time you assign one of the ? opcode values to an actual integer or string. This is much faster as you do not have to re-parse the predicate, and also means you avoid failing in incomprehensible ways if the user searches for nonsense like ]@attr. Borrowing from SQL, the syntax should be familiar:

g_autoptr(XbQuery) query = xb_query_new (silo, "components/component/id[text()=?]/..", &error);
xb_query_bind_str (query, 0, "gimp.desktop", &error);

The second feature makes the caller jump through some hoops, but hoops that make things faster: Indexed queries. As it might be apparent to some, libxmlb stores all the text in a big deduplicated string table after the tree structure is defined. That means if you do component then we only store just one string! When we actually set up an object to check a specific node for a predicate (for instance, text()='fubar' we actually do strcmp("fubar", "component") internally, which in most cases is very fast…

Unless you do it 10 million times…

Using indexed strings tells the XbMachine processing the predicate to first check if fubar exists in the string table, and if it doesn’t, the predicate can’t possibly match and is skipped. If it does exist, we know the integer position in the string table, and so when we compare the strings we can just check two uint32_t’s which is quite a lot faster, especially on ARM for some reason. In the case of fwupd, it is searching for a specific GUID when returning hardware results. Using an indexed query takes the per-device query time from 3.17ms to about 0.33ms – which if you have a large number of connected updatable devices makes a big difference to the user experience. As using the indexed queries can have a negative impact and requires extra code it is probably only useful in a handful of cases. In case you do need this feature, this is the code you would use:

xb_silo_query_build_index (silo, "component/id", NULL, &error); // the cdata
xb_silo_query_build_index (silo, "component", "type", &error); // the @type attr
g_autoptr(XbNode) n = xb_silo_query_first (silo, "component/id[text()=$'test.firmware']", &error);

The indexing being denoted by $'' rather than the normal pair of single quotes. If there is something more standard to denote this kind of thing, please let me know and I’ll switch to that instead.

The third feature is: “gaming mouse” and still get results that mention games, game and Gaming. This is also how you can search for words like Kongreßstraße which matches kongressstrasse. In an ideal world stemming would be computationally free, but if we are comparing millions of records each call to libstemmer sure adds up. Adding the stem() XPath operator took a few minutes, but making it usable took up a whole weekend.

The query we wanted to run would be of the form id[text()~=stem('?') but the stem() would be called millions of times on the very same string for each comparison. To fix this, and to make other XPath operators faster I implemented an opcode rewriting optimisation pass to the XbMachine parser. This means if you call lower-case(text())==lower-case('GIMP.DESKTOP') we only call the UTF-8 strlower function N+1 times, rather than 2N times. For lower-case() the performance increase is slight, but for stem it actually makes the feature usable in gnome-software. The opcode rewriting optimisation pass is kinda dumb in how it works (“lets try all combinations!”), but works with all of the registered methods, and makes all existing queries faster for almost free.

One common question I’ve had is if libxmlb is supposed to obsolete appstream-glib, and the answer is “it depends”. If you’re creating or building AppStream metadata, or performing any AppStream-specific validation then stick to the appstream-glib or appstream-builder libraries. If you just want to read AppStream metadata you can use either, but if you can stomach a binary blob of rewritten metadata stored somewhere, libxml is going to be a couple of orders of magnitude faster and use a ton less memory.

If you’re thinking of using libxml in your project send me an email and I’m happy to add more documentation where required. At the moment libxmlb does everything I need for fwupd and gnome-software and so apart from bugfixes I think it’s basically “done”, which should make my manager somewhat happier. Comments welcome.

Powered by WPeMatico

Share Button