Finer granularity for OS/2 DLL dependency generator

[dmik]

Currently, the OS/2 DLL dependency generator detects only DLL names for Requires/Provides but not the functions which are imported or exported. However, sometimes the import/export list of a DLL changes w/o changing its name. The most frequent case is addition of new exports.

Imagine a situation: LIB.DLL gets a new export FOO. This lib is released to a beta repository (not available for everyone) with Provides: LIB.DLL added by the dependency generator to its RPM. The same Provides existed for a previous version of this DLL before FOO was exported. A developer then builds some APP that requires LIB.DLL and happens to use the new export FOO just after rebuild (no change to the APP source is made) because it's an override (i.e. FOO exe was imported from some other lib before). The generator adds Requires: LIB.DLL to APP's RPM after analyzing the IMPORTS table of its EXE/DLL files. This APP is then released to a public repository. And if the user installs or updates this APP, the LIB.DLL dependency will be satisfied by an earlier version of LIB.DLL which lacks the FOO export, resulting in export FOO not found at runtime. A real-life example of this scenario is #257.

Note that this mostly applies to cases when new exports are added to a DLL because if there is a change or removal, the DLL becomes non-backward compatible which requires to change its name (and when the name changes, the dependency generator will update the dependency forcing the user to update the DLL as well).

A proposed solution is to increase granularity of the Provides/Requires records by adding not only the DLL name but the export name as well — using the parenthesis notation just like other generators (e.g. perl, python, pkg-config etc) do. So in the above imaginary case, the generator should add the following for the LIB.DLL RPM:

Provides: LIB.DLL
Provides: LIB.DLL(FOO)

And the following for the APP RPM:

Requires: LIB.DLL(FOO)

instead of just LIB.DLL as it does now.

This way, any potential problems related to export/import mismatches will be solved automatically and for all possible combination of package dependencies.

This is relatively easy to implement as all imports and exports are collected in the respective sections of EXE and DLL files in binary form (so it's fast to extract this information). The only potential problem I see is C++ DLLs which export huge amounts of C++ symbols (i.e. the Qt DLLs contain thousands of exports). But it doesn't sound too serious per se as RPM is designed to handle thousands of dependencies.

[dmik]

There is, of course, a way to manually add dependencies to a specific version of LIB.DLL, like Requires: LIB.DLL >= x.y.z provided that version x.y.z of LIB.DLL contains FOO, but this route is error-prone. So an automated solution is required.

[Yuri Dario]

I don't agree with such kind of automatic dependency generation.

If a new function is added to a DLL, the dll must change its name. Also we are unable to catch ABI changes this way, so we are still open to issues.
And also AOO exports thousands of functions and not for a few dlls only. I think we are going to grow the dependancy tree enormously this way.

[dmik]

Why change its name? This will require to either rebuild all apps using it or provide a forwarder. It's okay (and it's what we already do) when there is a non-backward compatible change. But simply adding a function is almost always backward compatible so there is no need to rename the dll and rebuild apps/provide forwarders. Everything will work perfectly well with the old name given that a proper version of the DLL is installed.

ABI changes other than adding a completely new function almost always mean changing function signatures. For C++ it is done automatically in most cases. For plain C it's the author's responsibility to rename the function if it's signature changes. And if the function is renamed, the automatic dependency generator will catch it.

Regarding the dependency tree. While I think that RPMDB is capable of handling thousands of dependencies, I agree that C++ may theoretically impose some problems (I already wrote about that and I thought exactly about AOO and also about Qt). But given that the OS/2 kernel itself feels pretty much ok with them, I guess RPM will also do. And as a last resort, If we are not satisfied with real-life tests, we may simply disable this generator for C++ symbols.

So I still think we should do it.

[Silvan Scherrer]

to be honest I doubt we will ever do that. So I close it here.