Corruption of VLC window

[KO Myung-Hun]

Hi/2.

While playing a video on an embed video mode, if selecting View > Playlist, the playlist window is corrupted.

And after a playing a video on an embed video mode, resizing a VLC main window causes a corruption of a logo, VLC corn.

I think, these are problems of the same cause.

[Dmitry A. Kuminov]

This behavior I could reproduce with tests/embedded in r1004. Investigating.

[Dmitry A. Kuminov]

This is fixed in r1008. We were accidentally using wrong HWNDs when detecting the visibility state of widgets with native HWNDs that were top-level ones but then were reparented and became child widgets.

[Dmitry A. Kuminov]

There is one more problem. If the external program keeps to write to the HWND that was moved to the bottom of the Z-stack (in terms of Qt) instead of hiding it, the writes are still visible. This is because other widgets in this Z-stack are "virtual", they are not real HWNDs and can't shadow the real one which is behind them.

r1010 demonstrates this behavior (press Ctrl-P then Ctrl-L).

[Dmitry A. Kuminov]

The problem persists even in QT_USE_NATIVE_WINDOWS=1 mode (where each widget is backed up by a native HWND) -- because we don't use WS_CLIPSBLINGS and WS_CLIPCHILDREN (for the reasons described in qwidget_pm.cpp). Instead of using these flags, we maintain clipping completely our own when we paint to the respective HWNDs (by manually excluding their overlapping siblings and children from the clip region).

However, when the external application paints our HWND, it doesn't know these rules and assumes all necessary clipping is already done.

[Dmitry A. Kuminov]

Since system clipping is not used, the application painting to the widget directly bypassing Qt needs to call our special routine, qt_WinProcessWindowObstacles(), that clips out parts of the widget obscured by other widgets -- there is just no other way (using WS_CLIPSIBLINGS and WS_CLIPCHILDREN in Qt is not a solution because that would break support for non-rectangular child widgets in Qt). This function is actually exported by QtGui4.dll from the very beginning is rather easy to use. I will describe that in the next comment.

The problem in comment:3 is solved in r1012. Note that this still requires qt_WinProcessWindowObstacles() to be used. The change just makes sure that all obscuring widgets get native windows so that they are recognized by qt_WinProcessWindowObstacles().

[Dmitry A. Kuminov]

In r1013, I made this function an APIENTRY export, so that it's very easy to use it from external applications. Here is a snippet:

#define INCL_WINWINDOWMGR
#define INCL_GPIREGIONS
#include <os2.h>

// although the prototype is available in <QWidget>, external applications
// may need to declare it manually and link to QtGui4.lib directly, or even
// query it from QtGui4.dll dynamically at runtime
LONG APIENTRY qt_WinProcessWindowObstacles(HWND hwnd, RECTL *prcl, HRGN hrgn,
                                           LONG op, LONG flags);

void externalPaint(HWND hwnd)
{
    // hwnd is the value returned by QWidget::winId()

    RECTL rcl;
    WinQueryWindowRect(hwnd, &rcl);
    HPS hps = WinGetPS(hwnd);

    HRGN hrgn = GpiCreateRegion(hps, 1L, &rcl);
    ULONG rc = qt_WinProcessWindowObstacles(hwnd, NULL, hrgn, CRGN_DIFF,
                                            0 /* PWO_Default */);
    if (rc == RGN_RECT || rc == RGN_COMPLEX) {
        HRGN hrgnOld;
        GpiSetClipRegion (hps, hrgn, &hrgnOld);
        hrgn = hrgnOld;

        // Paint to hps using regular PM and GPI calls
    }

    GpiDestroyRegion (hps, hrgn);

    WinReleasePS (hps);
}

KO, you need to update your kva driver to avoid the original problem described in this ticket. Note that you don't need to set any WS_CLIP* on the handle obtained from Qt on your own (most likely, they won't have any effect).

Note that this function is now documented in the QWidget class section of the Qt documentation. What I didn't like though is that, due to the dumbness of qdoc (which doesn't understand many simple things like function modifiers or unnamed enums) and the fact that docs are always rebuilt completley (which takes just a bulk of time), it took many hours to put it there.

[KO Myung-Hun]

dmik,

kva plugin should not depend on any interfaces including Qt.

So, it's not a good way to insert Qt codes into kva plugin.

Instead, would you mind using WinBeginEnumWindows()/WinGetNextWindow()/WinEndEnumWindows() APIs in Qt to detect external obstacles ?

[Dmitry A. Kuminov]

KO, you seem to not get it right. When you do WinGetPS() on an arbitrary HWND from the KVA plugin and do *not* set up your own clipping, you obviously assume that PM does the right clipping for you (based on WS_CLIPSIBLINGS and WS_CLIPCHILDREN flags of this HWND and its ancestors).

However, all HWNDs that originate from Qt do *not* have WS_CLIPSIBLINGS and WS_CLIPCHILDREN flags which means that PM will not set up any clipping in response to WinGetPS() on the Qt's HWND and you will always get a completely unclipped PS that lets you paint over any obstacles of this HWND.

We cannot intercept the WinGetPS() calls in Qt to set up the proper clipping for you (well, technically, nothing is impossible, but that would be very complex and totally not worth it). The only thing we could do in Qt is use the CLIP flags but this is not going to happen because that would break non-rectangular widgets in Qt (as I said earlier). This means that the only way to go for now is that your driver "detects external obstacles" and sets up proper clipping when it paints on the Qt's HWND.

Since that would be a non-trivial task, I have provided a magic export that nicely does that for you. This is not a dependency on the Qt interfaces at all, qt_WinProcessWindowObstacles() is a pure PM extension, it does not rely on Qt internals. It just fixes what I consider to be a PM bug. This API should work perfectly well for any HWND, not necessarily the one from Qt.

So I see no problem for your code to depend on it. Just load this export from QTGUI4.DLL, if it's there, and you are done (it will actually be in memory most of the time anyway, since most users will use the Qt interface of VLC). If not, well, simply assume that PM does the clipping for you, as before.

[Dmitry A. Kuminov]

It may be a good idea to move this and a number of other PM extensions from the Qt DLLs to a separate DLL though (to break the Qt DLL name dependency) but we have no time for that. If you manage to do that, we will kindly accept your patches (created #241 for that).

[Dmitry A. Kuminov]

Another option for you would be to patch the Qt4 frontend of VLC to avoid situations when the video widget is partly or fully obscured. For example, by hiding it when showing the play list (instead of just putting behind the play list widget in the widget stack that manages them). This way, no over-painting will take place and more over, you will be able to detect the hidden state from the KVA plugin and stop rendering frames, etc.

[KO Myung-Hun]

kva plugin does not call WinGetPS() at all, and neither libkva does. But libkva gets ps handle only in WM_ERASEBACKGROUND.

And kva plugin adds WS_CLIPCHILDREN to the window style of its parent window. So there is no problem whether or not Qt's HWND has that window style.

In addition, the problem still occurs regardless of adding WS_CLIPCHILDREN.

When tested with X-Ray, switching from a video window to a playlist window causes a playlist window and a video window are displayed partially with its window id.

And as you know, while playing, resizing a VLC main window in a playlist mode causes a video window to cover all the a playlist mode.

This is a clipping problem ? or a Z-stack problem ? or the same thing ?

BTW, the strange one thing is this problem occurs even after stopping playing a movie.

This is the time when a kva window and a video widget window are closed.

However, these problems have gone when using QT_USE_NATIVE_WINDOWS=1, although another problem that control buttons does not work occurs.

Any idea ?

[KO Myung-Hun]

About QT_USE_NATIVE_WINDOWS=1, it does not work with dive. And dive mode of kva use WinGetPS().

[Dmitry A. Kuminov]

Let's not mix all things together. From the VLC's Qt GUI perspective, there are two different use cases:

1. When video window auto-resize is turned on.
2. When video window auto-resize is turned off.

In case of [1], the video widget gets hidden when the play list is activated or when the video is stopped. However, due to some bugs in Qt 4.6.3, it was not actually hidden (unless you specified QT_USE_NATIVE_WINDOWS=1) so it was still visible. The current Qt SVN fixes all these problems, so that it works correctly even w/o QT_USE_NATIVE_WINDOWS=1.

In case of [2], the video widget does not get hidden, it just gets placed behind the play list widget in the widget stack. I don't know why as I didn't look close at VLC sources. And since it is not hidden, it requires overlying widgets to be clipped out when painting on it but nobody does that clipping (because of the lack of WS_CLIPSIBLING flags). So it's a pure clipping problem. Z-order of widgets should be correct.

Adding WS_CLIPCHILDREN to the parent doesn't actually make any sense at all because that would affect parent's clipping, not the video window's one. Only WS_CLIPSIBLINGS matters here but it must be set on all ancestors and their siblings to get the proper effect (and it's a bad idea to do it from KVA since that would interfere with Qt in many regards).

Regarding the control buttons, I don't see any problems here. Could you be more specific please?

BTW, do you use Qt built from the current SVN?

[Dmitry A. Kuminov]

I would also like to know who, how and when paints to the HWND in modes other than Dive.

[Dmitry A. Kuminov]

I've just checked again, the video widget actually *gets* hidden (both in Qt and in PM terms) both in cases 1 and 2. The only difference is that case 1 will normally cause the main window resize.

If you try to resize the window in case 2 after you switch to the play list view with Ctrl-L (or simply click on the main window's titlebar instead of resizing), the video widget will immediately go away and won't appear until you bring it in front again with Ctrl-L.

This makes me think that the guilty person is whoever actually paints to the video window. This person seems to completely ignore the fact that the window's parent (VideoWidget) is hidden and continues to paint over whatever it is at its former location (the play list in this case). Only triggering a complete window repaint (titlebar click or resize) causes it to notice that fact.

My guess is that it's a Dive/SDD bug (or what subsystem you use for painting, I don't know). I doubt we can work it around in Qt.

But you may still try to use qt_WinProcessWindowObstacles() on a HWND where actual painting goes on (no matter if you call WinGetPS yourself or not). This method will give you the proper clip region that you may then set on a painting surface using the methods of the subsystem you use for painting (Dive/SDD). May be this will work.

The fact that the problem even persists when you pause the video only confirms this guess since it looks like the buffer is still flushed by that subsystem.

P.S. Even your X-Ray will show you that the VideoWidget is hidden when you switch to the play list with Ctrl-L. So it's not a misinterpretation of the hidden state in Qt as one could think.

[KO Myung-Hun]

Replying to dmik:

Let's not mix all things together. From the VLC's Qt GUI perspective, there are two different use cases:

1. When video window auto-resize is turned on.
2. When video window auto-resize is turned off.

In case of [1], the video widget gets hidden when the play list is activated or when the video is stopped. However, due to some bugs in Qt 4.6.3, it was not actually hidden (unless you specified QT_USE_NATIVE_WINDOWS=1) so it was still visible. The current Qt SVN fixes all these problems, so that it works correctly even w/o QT_USE_NATIVE_WINDOWS=1.

I don't understand what you mean by auto-resize. This is a feature of VLC or of Qt ? How can I control it ?

In case of [2], the video widget does not get hidden, it just gets placed behind the play list widget in the widget stack. I don't know why as I didn't look close at VLC sources. And since it is not hidden, it requires overlying widgets to be clipped out when painting on it but nobody does that clipping (because of the lack of WS_CLIPSIBLING flags). So it's a pure clipping problem. Z-order of widgets should be correct.

Adding WS_CLIPCHILDREN to the parent doesn't actually make any sense at all because that would affect parent's clipping, not the video window's one. Only WS_CLIPSIBLINGS matters here but it must be set on all ancestors and their siblings to get the proper effect (and it's a bad idea to do it from KVA since that would interfere with Qt in many regards).

As you said, WS_CLIPCHILDREN is needed for a parent window to overpaint from a kva window.

It's not for Qt. If it is not set, some parent windows can clear a overlay color of a kva window.

Please remember that kva plugin is a module for a general PM program, not only for Qt.

Hmm... clipping problems... a solution of Qt4 to workaround a problem of PM, generates another problem.

Regarding the control buttons, I don't see any problems here. Could you be more specific please?

Ok. I meant 'Play', 'Stop', and etc buttons by the control buttons.

When I set QT_USE_NATIVE_WINDOWS=1, those buttons do not work even though I clicked on them.

BTW, do you use Qt built from the current SVN?

No, I'm using 4.6.3GA.

I have no time to build Qt from SVN.

I'll be glad if you provide me the latest binary.

[KO Myung-Hun]

Replying to dmik:

I would also like to know who, how and when paints to the HWND in modes other than Dive.

SNAP and WarpOverlay! with a overlay feature using a color key.

[KO Myung-Hun]

I'll try to use qt_WinProcessWindowObstacles() directly not using Qt4 DLLs.

Then, I'll inform you of the result.

[KO Myung-Hun]

I've tested the codes using qt_WinProcessWindowObstacles(). But nothing changed.

The following is the codes modified by me. Maybe, I was using wrong codes ?

static HPS qt_display_ps()
{
    static HPS displayPS = 0;
    
    if (displayPS == 0 )
        displayPS = WinGetScreenPS(HWND_DESKTOP);
    return displayPS;
}

static int qt_display_width()
{
    // display resolution is constant for any given bootup
    static LONG displayWidth = 0;
    if (displayWidth == 0)
        displayWidth = WinQuerySysValue(HWND_DESKTOP, SV_CXSCREEN);
    return displayWidth;
}

static int qt_display_height()
{
    // display resolution is constant for any given bootup
    static LONG displayHeight = 0;
    if (displayHeight == 0)
        displayHeight = WinQuerySysValue(HWND_DESKTOP, SV_CYSCREEN);
    return displayHeight;
}

static void qt_WinQueryClipRegionOrRect(HWND hwnd, HRGN hrgn)
{
    RECTL rcl;
    WinQueryWindowRect(hwnd, &rcl);

    HPS hps = qt_display_ps();
    GpiSetRegion(hps, hrgn, 1, &rcl);
    if (WinQueryClipRegion(hwnd, 0) != QCRGN_NO_CLIP_REGION) {
        HRGN hrgnTemp = GpiCreateRegion(hps, 0, NULL);
        WinQueryClipRegion(hwnd, hrgnTemp);
        GpiCombineRegion(hps, hrgn, hrgnTemp, hrgn, CRGN_AND);
        GpiDestroyRegion(hps, hrgnTemp);
    }
}

enum PWOFlags {
    PWO_Children = 0x01,
    PWO_Siblings = 0x02,
    PWO_Ancestors = 0x04,
    PWO_Screen = 0x08,
    PWO_TopLevel = 0x80000000,
    PWO_Default = 0 /*PWO_Children | PWO_Siblings | PWO_Ancestors | PWO_Screen*/,
};

static LONG qt_WinProcessWindowObstacles(HWND hwnd, RECTL *prcl, HRGN hrgn,
                                         LONG op, LONG flags)
{
    if (flags == 0)
        flags = PWO_Children | PWO_Siblings | PWO_Ancestors | PWO_Screen;

    HPS displayPS = qt_display_ps();

    SWP swpSelf;
    WinQueryWindowPos(hwnd, &swpSelf);

    RECTL rclSelf = { 0, 0, swpSelf.cx, swpSelf.cy };
    if (prcl)
        rclSelf = *prcl;

    HRGN whrgn = GpiCreateRegion(displayPS, 0, NULL);

    LONG cmplx = RGN_NULL;
    HWND relative;
    SWP swp;

    BOOL cmplxChanged = FALSE;

    // first, process areas placed outside the screen bounds
    if (flags & PWO_Screen) {
        RECTL rclScr = { 0, 0, qt_display_width(), qt_display_height() };
        WinMapWindowPoints(HWND_DESKTOP, hwnd, (PPOINTL) &rclScr, 2);
        // rough check of whether some window part is outside bounds
        if (rclSelf.xLeft < rclScr.xLeft ||
            rclSelf.yBottom < rclScr.yBottom ||
            rclSelf.xRight > rclScr.xRight ||
            rclSelf.yTop > rclScr.yTop) {
            GpiSetRegion(displayPS, whrgn, 1, &rclSelf);
            HRGN hrgnScr = GpiCreateRegion(displayPS, 1, &rclScr);
            // substract the screen region from this window's region
            // to get parts placed outside
            GpiCombineRegion(displayPS, whrgn, whrgn, hrgnScr, CRGN_DIFF);
            GpiDestroyRegion(displayPS, hrgnScr);
            // process the region
            if (hrgn != NULLHANDLE) {
                cmplx = GpiCombineRegion(displayPS, hrgn, hrgn, whrgn, op);
                cmplxChanged = TRUE;
            } else {
                WinValidateRegion(hwnd, whrgn, FALSE);
            }
         }
    }

    // next, go through all children (in z-order)
    if (flags & PWO_Children) {
        relative = WinQueryWindow(hwnd, QW_BOTTOM);
        if (relative != NULLHANDLE) {
            for (; relative != HWND_TOP; relative = swp.hwndInsertBehind) {
                WinQueryWindowPos(relative, &swp);
                // skip if hidden
                if (!(swp.fl & SWP_SHOW))
                    continue;
                // rough check for intersection
                if (swp.x >= rclSelf.xRight || swp.y >= rclSelf.yTop ||
                    swp.x + swp.cx <= rclSelf.xLeft ||
                    swp.y + swp.cy <= rclSelf.yBottom)
                    continue;
                // get the bounds (clip region or rect)
                qt_WinQueryClipRegionOrRect(relative, whrgn);
                // translate the region to this window's coordinate system
                POINTL ptl = { swp.x, swp.y };
                GpiOffsetRegion(displayPS, whrgn, &ptl);
                // process the region
                if (hrgn != NULLHANDLE) {
                    cmplx = GpiCombineRegion(displayPS, hrgn, hrgn, whrgn, op);
                    cmplxChanged = TRUE;
                } else {
                    WinValidateRegion(hwnd, whrgn, FALSE);
                }
            }
        }
    }

    HWND desktop = WinQueryDesktopWindow(0, 0);
    HWND parent = WinQueryWindow(hwnd, QW_PARENT);

    // next, go through all siblings placed above (in z-order),
    // but only if they are not top-level windows (that cannot be
    // non-rectangular and thus are always correctly clipped by the system)
    if ((flags & PWO_Siblings) && parent != desktop) {
        for (relative = swpSelf.hwndInsertBehind;
              relative != HWND_TOP; relative = swp.hwndInsertBehind) {
            WinQueryWindowPos(relative, &swp);
            // skip if hidden
            if (!(swp.fl & SWP_SHOW))
                continue;
            // rough check for intersection
            if (swp.x >= swpSelf.x + rclSelf.xRight ||
                swp.y >= swpSelf.y + rclSelf.yTop ||
                swp.x + swp.cx <= swpSelf.x + rclSelf.xLeft ||
                swp.y + swp.cy <= swpSelf.y + rclSelf.yBottom)
                continue;
            // get the bounds (clip region or rect)
            qt_WinQueryClipRegionOrRect(relative, whrgn);
            // translate the region to this window's coordinate system
            POINTL ptl = { swp.x - swpSelf.x, swp.y - swpSelf.y };
            GpiOffsetRegion(displayPS, whrgn, &ptl);
            // process the region
            if (hrgn != NULLHANDLE) {
                cmplx = GpiCombineRegion(displayPS, hrgn, hrgn, whrgn, op);
                cmplxChanged = TRUE;
            } else {
                WinValidateRegion(hwnd, whrgn, FALSE);
            }
        }
    }

    // last, go through all siblings of our parent and its ancestors
    // placed above (in z-order)
    if (flags & PWO_Ancestors) {
        POINTL delta = { swpSelf.x, swpSelf.y };
        while (parent != desktop) {
            HWND grandParent = WinQueryWindow(parent, QW_PARENT);
            if (!(flags & PWO_TopLevel)) {
                // When PWO_TopLevel is not specified, top-level windows
                // (children of the desktop) are not processed. It makes sense
                // when qt_WinProcessWindowObstacles() is used to clip out
                // overlying windows during regular paint operations (WM_PAINT
                // processing or drawing in a window directly through
                // WinGetPS()): in this case, top-level windows are always
                // correctly clipped out by the system (because they cannot be
                // non-rectangular).
                if (grandParent == desktop)
                    break;
            }

            WinQueryWindowPos(parent, &swp);
            delta.x += swp.x;
            delta.y += swp.y;
            for (relative = swp.hwndInsertBehind;
                 relative != HWND_TOP; relative = swp.hwndInsertBehind) {
                WinQueryWindowPos(relative, &swp);
                // skip if hidden
                if (!(swp.fl & SWP_SHOW))
                    continue;
                // rough check for intersection
                if (swp.x - delta.x >= rclSelf.xRight ||
                    swp.y - delta.y >= rclSelf.yTop ||
                    swp.x - delta.x + swp.cx <= rclSelf.xLeft ||
                    swp.y - delta.y + swp.cy <= rclSelf.yBottom)
                    continue;
                // get the bounds (clip region or rect)
                qt_WinQueryClipRegionOrRect(relative, whrgn);
                // translate the region to this window's coordinate system
                POINTL ptl = { swp.x - delta.x, swp.y - delta.y };
                GpiOffsetRegion(displayPS, whrgn, &ptl);
                // process the region
                if (hrgn != NULLHANDLE) {
                    cmplx = GpiCombineRegion(displayPS, hrgn, hrgn, whrgn, op);
                    cmplxChanged = TRUE;
                } else {
                    WinValidateRegion(hwnd, whrgn, FALSE);
                }
            }
            parent = grandParent;
        }
    }

    GpiDestroyRegion(displayPS, whrgn);

    if (hrgn != NULLHANDLE && cmplxChanged == FALSE) {
        // make sure to return the original complexity of the region
        RECTL rclDummy;
        cmplx = GpiQueryRegionBox(displayPS, hrgn, &rclDummy);
    }

    return cmplx;
}

And the painting codes to fill with a overlay color,

        case WM_ERASEBACKGROUND :
        {
            HPS     hpsFrame = ( HPS )mp1;
            PRECTL  prcl = ( PRECTL )mp2;

    HRGN hrgn = GpiCreateRegion(hpsFrame, 1L, prcl);
    ULONG rc = qt_WinProcessWindowObstacles(hwnd, NULL, hrgn, CRGN_DIFF,
                                            0 /* PWO_Default */);
    if (rc == RGN_RECT || rc == RGN_COMPLEX) {
        HRGN hrgnOld;
        GpiSetClipRegion (hpsFrame, hrgn, &hrgnOld);
        hrgn = hrgnOld;
            
        // Paint to hps using regular PM and GPI calls
            GpiCreateLogColorTable( hpsFrame, 0, LCOLF_RGB, 0, 0, NULL );
            WinFillRect( hpsFrame, prcl, g_ulKeyColor);
    }

    GpiDestroyRegion (hpsFrame, hrgn);

            return FALSE;
        }

[Dmitry A. Kuminov]

Ah, the color code, right. This is why the video window needs to be always black. Okay. I use KVA-DIVE anyway and all my descriptions relate to it so far. So let's sort out this mode first.

We have the following layout of windows in question (demonstrating their parent-child relationships):

[1] Qt VideoWidget
 |    this one is stacked with the Play List widget in VLC GUI
 |
 +--[2] Qt "stable" widget
     |    this one's HWND is returned by VideoWidget::request()
     |
     +--[3] KVA frame window
         |
         |
         +--[4] KVA client window
                  this is where all video painting actually goes)

Regarding your use of WS_CLIPCHILDREN. Your code sets this flag on window [2], to make sure that this window's paint procedure does not overpaint its children, [3] and [4]. I see. However, this flag should *not* be necessary, because window [2] is a Qt window and the Qt paint procedure knows how to clip out children even w/o WS_CLIPCHILDREN being set (thanks to qt_WinProcessWindowObstacles()). If it's *not* the case, please confirm that as this should be considered as a Qt bug then. Again, I'm talking about Dive here, can't experiment with the overlay mode ATM.

Regarding auto-resize. I'm talking about the VLC feature which you may turn on and off (Tools - Preferences - Interface - Resize interface video size). I've just checked it once more after enabling WS_CLIPCHILDREN and WS_CLIPSIBLINGS in my local Qt build (to make sure that WinGetPS() does the proper clipping w/o the need of qt_WinProcessWindowObstacles()) -- and nothing changed. I can still see the described effect: if auto-resize is OFF and I play the video, switching to Play List keeps the video frame above it until I either a) click on the window's title or b) resize the window. If auto-resize is ON, then the problem does not occur but only because the window is automatically resized when I switch to Play List and back to the video.

Regarding the Play/Stop and other control buttons. I see no problems with them. In all tested combinations it works as it should.

I think it makes no sense to further discuss things until you've got the new Qt build. As I said, a couple of related bugs were fixed while I was working on your tickets so we have the different ground now. I will give you the build now but please keep in mind that this is not how it is ought to be -- it is assumed that if you develop a Qt application (especially such a complex one), you have your own build of Qt which lets you instantly try various changes there (especially when debugging such complex problems). So please consider giving your PC a night to build Qt.

[Dmitry A. Kuminov]

The current Qt build is here: ​ftp://ftp.dmik.org/tmp/q/komh.zip. This build understands a special environment variable, QT_PM_FORCE_CLIPALL, which, if set, will cause Qt to create *all* widgets with WS_CLIPSIBLINGS and WS_CLIPCHILDREN flags turned on. Just for experimenting.

Regarding your code using qt_WinProcessWindowObstacles(), it seems to be correct. I still think it's an issue of the subsystem (Dive/SDD/WO) that doesn't notice the visible state change. To (dis)prove that, can you please try to disable the actual video overlaying by Dive/SDD/WO and only leave the code (like above) that fills the widget with the overlay color? Does it remove the problem or not?

Please also try all other cases with the new Qt build and report which problems you still see. Please mention the following for each case:

1. Mode (Dive/SDD/WO).
2. If QT_USE_NATIVE_WINDOWS=1 is set or not.
3. If Qt_PM_FORCE_CLIPALL=1 is set or not.
4. The exact course of action, e.g.:
   1. Start VLC with preference A turned on, preference B turned off
   2. Start playing .avi file
   3. Press this button.
   4. Open that window.
   5. Etc.

Otherwise it's really hard to understand what you mean. I don't have much time to debug this, unfortunately.

[KO Myung-Hun]

As I said many times, kva plugin is not only for Qt. It can be used with any other PM apps. Because VLC itself is a very-well-modularized-and-structured library. So assumption that it will be used with Qt only, is wrong. Actually, mozilla plugin uses vlc library.
So the fact that Qt does not require WS_CLIPCHILDREN because it can do the equivalent thing without it, is not important. There is no need to discuss this any more.

It's so terrible that I should consider a bug of Qt itself while developing a Qt app, moreover debug it by myself.

And I think it would be better that you supply nightly builds instead of letting individual programmer spend their night time with PC's noise and electronic-magnetic waves.

In addition, our electric-fee is not cheap.

Anyway, I'll test with your latest Qt and report the result to you as you want.

BTW, Time seems to be lack all the time. That's too bad. TT

Let's cheer ourselves up. ^_

[Dmitry A. Kuminov]

You seem to not understand me again. Of course, I speak about VLC *and* Qt here, since Qt is my primary concern. And I don't mean that you must remove WS_CLIPCHILDREN from your code, I mean that in case of the Qt front end, adding or removing this flag should not make any difference. It is more a debugging hint, something for you to try out. If you don't want to participate in debugging Qt (and eventually making it better), I'm not going to force you, of course.

Regarding bugs. I could say exactly the same: "it's so terrible that I should consider a bug of VLC itself while developing Qt, moreover debug it by myself". But I didn't say that and instead I debugged VLC myself. JFYI.

I can understand your unwillingness to deal with Qt bugs. But I physically cannot do everything myself and I need help (the Qt project is very huge and there is also a number of other projects of a comparable size on my plate). The key thought here is that if you don't participate, you loose too. This is our reality. Whether you like it or not.

Regarding nightly builds. We simply don't have hardware for doing that ATM (a single build of Qt takes about 8 hours on a dual core machine and this machine obviously can't be used for anything else during that time). If we ever get such an opportunity, we will set up automatic builds. It's in our todo list. However, these builds will not help us to debug cases like this (because it requires instant changes and recompiling).

Other than that, thank you for reporting. But please understand that just letting us know about the problem is not always enough to have it solved.

[Dmitry A. Kuminov]

Okay, here is what I found after the chat with Myung-Hun. "GA" below is the 4.6.3 release, "TEST" is the SVN build (link above). He has SNAP 3.18 (some older build, not the eCS level). "Problem #203" is the original problem of this ticket, i.e. switching from the video view to the play list leaves the video view on top. All related to the VLC GUI.

1. In the default environment, TEST shows corruption of many (all?) widgets, including the main window's menu and so on.

2. In the default environment, GA doesn't show this menu corruption, but shows problem #203.

3. Setting QT_PM_NO_DIVE=1 solves menu corruption in TEST. It also shows no visible problems at all, i.e. all works great, and no problem #203 even if auto-resize is turned off.

4. Setting QT_PM_NO_DIVE=1 does not change anything in GA. Still the same problem #203.

I have the following conclusions so far:

1. Something between GA and TEST breaks DIVE support in Qt which causes menu corruption.

2. TEST actually fixes problem #203 on SNAP completely.

3. Part of problem #203 is still present in PANORAMA (when you need to click the title bar or resize the window to make it noticed that the video widget is hidden and should be not painted over).

This needs to be sorted out before the release. Especially item 1.

[Dmitry A. Kuminov]

Okay, I can confirm problem 1. These two settings trigger the bug in *all* Qt applications under both Panorama and SNAP:

set QT_PM_DIVE=BLIT
set QT_USE_NATIVE_WINDOWS=1

The first forces DIVE mode (which is OFF on Panorama by default), the second forces native window creation per each widget (which is also OFF by default).

[Dmitry A. Kuminov]

Fixed the problem described in comment:27. See r1033 for details. In short, it was a misalignment of dirty regions of child widgets with native HWNDs. The problem didn't show up before r1012 and therefore was not visible in Qt 4.6.3.

Myung-Hun, I uploaded the fixed QtGui4.dll as ​ftp://ftp.dmik.org/tmp/q/komh_2.zip. Please test and report back.

[KO Myung-Hun]

Ok. I've tested.

All is well except DIVE mode with auto-resize off.

Switching from a video window to a playlist window, a video window does not disappear, but resizing a main window, recovers this problem, regardless of any QT_xxx env. var. settings.

The other things than this, work as expected.

[Dmitry A. Kuminov]

Okay, good.

I know what causes the remaining problem. Our custom qt_WinSetWindowPos() procedure (that deals with non-rectangular windows again) calls the original WinSetWindowPos() API with the SWP_NOREDRAW flag set to avoid sending WM_PAINT messages to the widgets whose parts are invalidated by the operation.

However, it turns out that SWP_NOREDRAW also suppresses sending WM_VRNDISABLED/WM_VRNENABLED messages to the affected windows to inform them about changes to their visible regions. I didn't know that. And, frankly, I find it not logical -- asking "don't redraw" doesn't actually mean "don't inform about changes in what's visible". These are two distinct entities (areas to redraw are controlled by the update region; the visible region just defines what parts are visible, they don't necessarily need updates).

Anyway, I'm not sure on how to overcome this bug-o-feature. The problem is that while I can detect visible regions of which HWNDs get changed, I don't know which widgets asked to send them WM_VRNDISABLED/WM_VRNENABLED notifications (this is done WinSetVisibleRegionNotify() and there is no WinSetVisibleRegionNotify() counterpart).

[Dmitry A. Kuminov]

After thinking on it more, I decided to disable the native window masking code at all. There are several reasons for that:

1. Normally, native windows for child widgets are not created (it covers 99% of Qt applications). In this case, all masking (as well as transparency) is done by Qt itself, the native masking code is not involved at all.

2. Known exceptions that need native windows are video players but since they can't properly work in native masking mode due to lacking EM_VRNDISABLED/WM_VRNENABLED notifications, it makes little sense to work in this mode. Especially taking into account that these applications don't need fancy masks for the widget displaying the video anyway.

All the relevant code is guarded with the QT_PM_NATIVEWIDGETMASK compiler macro which is not defined by default, see r1034.

If/when we face an application that really needs native window masks, we will come back to this issue and try to fix it.

Ah, and since WM_VRNDISABLED/WM_VRNENABLED now works as expected, I enabled some missing bits related to its handling (r1035) and the last problem from comment:29 is gone now. The new DLL is uploaded as ​ftp://ftp.dmik.org/tmp/q/komh_3.zip, please check it.

And note also that after r1034 Qt now *sets* WS_CLIPSIBLINGS | WS_CLIPCHILDREN again on all native windows it creates which means that you don't need to use qt_WinProcessWindowObstacles() in your code. More over, this routine is not exported at all in the default !QT_PM_NATIVEWIDGETMASK mode.

[KO Myung-Hun]

Ok, it works well. Thanks for your hard works.

FWIW, would you mind testing following codes ?

WinLockWindowUpdate( HWND_DESKTOP, hwnd );
WinSetWindowPos( hwnd, hwndInsertBehind, x, y, cx, cy, fl );
WinSetWindowPos( hwnd, hwndInsertBehind, x, y, cx, cy, fl | SWP_NOREDRAW );
WinLockWindowUpdate( HWND_DESKTOP, NULLHANDLE );

[Dmitry A. Kuminov]

Thanks for the hint. Thought about that but it needs experimenting. Not critical for now. Created #243 for it. And thanks for testing.