Lecture 1: Windows graphical interfaces

Recently I was asked to lecture in my company, Netformx, about WPF. My goal was to make this lecture different from what you usually may see in WPF courses and events – instead of saying good sounding buzzwords and showing nice graphics, I intended to explain about how WPF is different from other available Microsoft technologies, and why it looks as it looks.  So I started to gather materials to base some technical background before digging deep into WPF, and very soon I realized it can be a standalone topic, justifying a separate lecture. Therefore, my first lecture will be called:

Windows graphical interfaces

Since 1985, when first version of Microsoft version released and up today, Microsoft released two main graphical interfaces: GDI and DirectX. Different frameworks such as a WPF are using these interfaces under the hood. In this lecture, we will discuss about each of them, and learn about appliances and limitations.

GDI

Graphic Device Interface (or GDI in short) was introduced back in 1985 with the first version of Microsoft Windows.  The idea behind GDI is to create an abstraction of physical device, so calling application doesn’t have to adjust its code for specific device. In other words, the same code that using GDI will be able to draw graphics on any GDI compatible device – and it will look the same each time. The device can be even a printer – and the calling application will not notice any difference. Microsoft called to that abstraction as “Device Context” – or DC in short.

So, GDI has two distinct worlds: the physical and the logical. Applications know about logical world, and telling to GDI what they want to draw, using logical GDI objects (such as pens, brushes, fonts, palettes, and bitmaps). Internally, Windows using GDI to draw its windows (frames) and common controls, such as message box. Any control based on GDI, consuming GDI objects. For example, when I opened a new empty window in GDI application I created, it consumed 24 GDI objects. Adding a button with text inside that window consumed additional 4 GDI objects. You can monitor how much GDI objects consumed by each process by opening Task Manager (by default, GDI Objects column is hidden). There is a limit of GDI objects, available for each process (the limit is various for different Windows versions). After the limit is reached, no more windows or controls can be created.

Next step after GDI receiving a call from application with logical objects, logical objects been converted to device specific physical objects, using device driver. This process called “rasterizing”.  During rasterizing, requested (logical) color is mapped to available colors, fonts are matched to the best available fonts, and patterns are prepared for output. Some parts of realization process handled by device driver, other handled by GDI itself.  The video driver is free to use any combination of CPU and GPU processing, however today only BitBlt API is hardware accelerated.

Early during boot (and Windows install) when no manufacturer-specific driver is available, the GDI does perform all rendering in software and writes to the framebuffer, which is just the memory area which feeds the GPU RAMDAC and mapped into the CPU address space. The framebuffer is stored in one of several well-known formats (defined by VESA). During this phase, you can notice that graphics is extremely slow (in selecting a region, try to open and scroll list of countries – it barely moving). This is because there is no video driver yet, and everything is done without any hardware acceleration.

GDI uses immediate rendering mode: the application repaints the area that becomes invalidated. On every invalidation, the WM_PAINT message is received by window. It is an application responsibility to handle that message, and to draw its state using GDI.

By maintaining the two separate but linked spaces, logical for the applications and physical for the devices, GDI creates a device-independent interface. Applications that make full use of the logical space and avoid device-specific assumptions can expect to operate successfully on any output device that comes down the turnpike. However, this advantage has a drawback: since GDI interface is a minimal subset of operations, which is common across different devices (video cards and printers), it does not expose advanced abilities of modern hardware such as visual effects, transforms, light sources, physics and more. As a result, any operation such this has to be calculated manually by application on CPU, before sending the result to GDI for rasterizing. This limitation (and inability to rasterize3D) makes GDI not suitable well for animations and rich visual effects.

GDI+

GDI+ presented in 2001 with release of Windows XP, as a successor of GDI. It resides in gdiplus.dll. GDI+ expanding GDI by adding the following abilities:

• 2D vector graphics. Vector graphics is based on graphic primitives like: lines, curves and shapes, all defined as a set of points on a coordinates system. Everything is relative regarding the origin of the coordinate system. Considering this, vector-based graphics can be scaled without losing details, which is a good thing, but rendering vector-based graphics demands a lot of CPU calculation (it is not hardware accelerated).
• Imaging. GDI+ can handle different types of bitmaps, including icons, PNG and JPEG formats and perform different image operations like: stretching and skewing.
• 2D anti-aliasing
• Floating point coordinates (needed for making resolution independence)
•  Alpha blend for colors (ARGB)

Overall, GDI+ is a little bit slower than the old GDI but provides better rendering quality and clean code.

Despite all the improvements of GDI, the same advantages (device abstraction) and disadvantages (device abstraction) are remaining. GDI+, just as GDI is good in drawing windows and buttons – but when it comes to animations or visual effects it very CPU consuming, since it very limited and (mostly) not hardware accelerated.

Note, that from 1985 and up today, during something like 25 years, GDI received only 1 update. I believe the reason for that is due the purpose of GDI to be a simple cross device graphical interface, Microsoft changes (almost) nothing in it, to keep it simple and generic.

DirectX

Back in 1994, while preparing for release of Windows 95, Microsoft realized that GDI is not suitable for high performance multimedia, such as games. Graphic cards (with 3D acceleration) was optimized to perform complex vector calculations, visual effects and more – but GDI didn’t provided access to these functions, in order to keep its interface simple and generic, common between video cards and printers. Game developers needed direct access to video card, sound device and mouse without any abstraction layers in the middle. Therefore in 1995, as part of Windows 95, Microsoft released a “bypass” – a set of interfaces for direct access, together called as “DirectX”. In scope of graphics, two most important interfaces are “Direct3D” – for rendering 3D graphics and DirectDraw (replaced by Direct2D in 2009) – for rendering 2D graphics.

DirectX (namely Direct3D) providing developers low level access to all functionality, the video cards capable to perform. In order to be updated, it keeps evolving together with graphical cards. Microsoft is all the time in touch with leading video cards manufacturers and constantly adjusting interfaces of DirectX, in order to expose newest functionality. From 1995 and until today, more than 25 versions of DirectX have been released.

Everything in Direct3D is hardware accelerated – many complex calculations performed by GPU, thus reducing the load on CPU. This comes at a price, however. When using DirectX, developers have to test whether current graphic card supports specific operation. If it not, they have to plan some fallback (like disabling animations, reducing quality and so on).  So, developers have to work harder, but they can do much more compared to GDI – and much faster.

Direct3D works in immediate mode: it follows the notation of “paint” calls to draw each frame.

DirectDraw is another graphical interface from a bundle called “DirectX”.  It designed to render 2D hardware accelerated graphics (also in immediate mode). DirectDraw allows direct access to video memory, hardware overlays, hardware blitters, and page flipping. Its video memory manager can manipulate video memory with ease, taking full advantage of the blitting and color decompression capabilities of different types of display adapters. Today it is deprecated, in favor of its successor Direct2D (D2D).

Direct2D is a latest graphic interface / technology released by Microsoft. It presented in 2009, together with DirectWrite (for a text layout) in Windows 7. According to Microsoft, the goal of D2D is to replace both DirectDraw and GDI+.  In this MSDN page (http://msdn.microsoft.com/en-us/library/ff729480%28v=vs.85%29.aspx), they recommend to consider writing new graphics code using Direct2D.

While GDI supports hardware acceleration for certain operations (BitBlt API), many GDI operations are bound to the CPU. Direct2D is layered on top of Direct3D, and takes full advantage of hardware acceleration provided by the GPU. If the GPU does not support the features needed for Direct2D, then Direct2D falls back to software rendering. Overall, Direct2D outperforms GDI and GDI+ in most situations.

Summary so far – Windows graphic APIs:

Desktop Window Manager

Before Windows Vista, the rendering was an easy thing: there was one shared memory buffer (which represented a desktop), and everybody wrote into it. Consider the example, where you have two open windows, one over other.  Moving upper window to new position will draw it at the new position, but since it shares a memory buffer with the second window, the second window will remain overwritten, as shown in the picture below:

In order to fix this issue, the bottom window has to handle a WM_PAINT message and to redraw itself. If the upper window will move constantly, the bottom window will have to redraw itself in high rate, and this will produce a “flicker” phenomenon. This is where the BitBlt API is very useful: using it, it is possible to implement “double buffer” technique and thus avoid “flicker” issue. In my previous article (http://evgenya.wordpress.com/2008/01/05/fast-rendering-images-in-c/), I demonstrated another uses for BitBlt for fast image rendering in C#.

Windows Vista presented several new features and these features caused to changing the way the graphics is rendered:

• Aero Glass – the semi-transparent look that Aero provides, with the blurry content behind the window frames, designed to allow the user to focus on the window itself, and not on what lies behind:

• Live thumbnails – previews provided on the taskbar:

• Flip3D – the ability to scroll open windows in the 3D space, for navigating between and selecting windows:

The component which handling all this staff called Desktop Window Manager, or DWM in short. Starting from Windows Vista, instead of writing into one shared memory buffer, each application writes to its own memory buffer. The application is not aware of that, and “thinks” that it writes to main buffer as usual. DWM intercepts these calls, and redirects them to off-screen buffer. This process called redirection. After that, DWM composes all this together in one single composed desktop. This change was necessary to support all features presented in Vista:

• Since every window is stored in different buffer, during composition the DWM can determine the color of each pixel. This is how frame transparency is achieved.
• Since the off-screen buffer is constantly updated by the application, the embedded rendering will be a dynamic representation of the application window and not a static rendering. This is how the live thumbnail previews, Windows Flip and Windows Flip 3D work in Windows Vista.

Moreover, since every window is retained and there is no shared memory buffer anymore, moving one window over other will not cause to the redraw of bottom window.

The DWM uses DirectX 9 to perform the function of compositing and rendering in the GPU, freeing the CPU of the task of managing the rendering from the off-screen buffers to the display. However, it does not affect applications painting to the off-screen buffers; depending on the technologies used for that, it might still be CPU-bound. DWM-agnostic rendering techniques like GDI are redirected to the buffers by rendering the UI as bitmaps. DWM-aware rendering technologies like WPF directly make the internal data structures available in a DWM-compatible format. The window contents in the buffers are then converted to DirectX textures.

The redirection process works different for different technologies:

GDI – rendered windows

• A system memory (in Win 7 – an aperture memory) surface the size of the window is allocated and associated with that window.
• A video memory surface, in the target DirectX pixel format, is allocated, also the size of the window.
• When an application retrieves the GDI DC of an HWND, it no longer is the DC of the primary video buffer, as it is in the non-composited, pre-DWM desktop.  Instead, the DC is a DC onto the allocated system memory surface.
• GDI operations on that DC then populate the system memory surface.
• The system, based on a number of variables, decides to update the video memory surface from the system memory surface at the “right times”.
• The video memory surface is now up-to-date with the application, and the compositor comes around and uses the video memory surface to composite the desktop from.

In GDI based windows, for each window allocated memory buffer (in Vista it is a system memory, in Win 7 – aperture memory). Since GDI uses its own format (GDI objects, etc), it renders as usual to its DC, which is a memory buffer. After that, the whole buffer moved as a single bitmap to video buffer of DWM (which is using DirectX pixel format).

DirectX – rendered windows

DirectX applications can natively render into the DirectX pixel format that the DWM expects. As such, DirectX applications only need a single window buffer to manage their redirection.  DirectX window redirection is handled by having the DirectX system, when it’s determining what surface to provide the app with to render to, make calls to the DWM in order to share a surface between the DirectX client application process, and the DWM process. WPF applications are DirectX applications, so the rendering process for them is the same.

GDI limitations under Windows Vista

At time of developing of Windows Vista, Microsoft found too difficult to combine the new graphical engine with GDI. As a result, under Windows Vista there is no at all hardware acceleration for GDI. This is why under the Vista, GDI uses system memory to store a buffer: GDI is being rendered utilizing the CPU completely in the operating system without any assistance or “acceleration” by the graphics hardware, and it requires an easily accessible cacheable copy of memory.

In addition, the design of GDI synchronization mechanism in Vista allows only a single application to hold a system-wide exclusive global lock:

In other words, if there are several GDI applications running at the same time on machine, one application will wait for other to perform rendering.

Both of these problems were fixed in Windows 7, through according to different sources GDI performance under Windows 7 is lower than under Windows XP.

I found this video, comparing performance of GDI under XP and Windows 7:

The following tables summarizes this lecture:

CriteriaTech\	Mode	Is hardware accelerated	DWM compatible
GDI	Immediate	Partly (non under Vista)	No
GDI+	Immediate	Partly (non under Vista)	No
D2D	Immediate	Yes	Yes
D3D	Immediate	Yes	Yes

CriteriaTech\	Uses system memory	Is resolution independent	Is allow direct access to video card	Supported in Windows XP?
GDI	Yes (GDI objects)	No	No	Yes
GDI+	Yes (GDI objects)	No	No	Yes
D2D	No	Yes	?	No
D3D	No	?	Yes	Yes

 

 

 

 

 

(as “?” I marked fields where I not 100% sure)

From the table above, we can learn that DirectX technology is far superior to a GDI one. Unlike GDI, DirectX is being constantly updated to reflect latest video hardware abilities. Under windows Vista, GDI has no hardware acceleration at all and suffers from GDI graphics stack bottleneck. Under rest of the operating systems, GDI has only BitBlt API hardware acceleration.

Coming Next:

Lecture 2- Windows UI Frameworks and introduction to WPF

References

1. http://stackoverflow.com/questions/6591227/how-gdi-gdi-works-without-opengl-or-directx
2. http://dotnetfacts.blogspot.com/2008/04/gdi-windows-rendering-api.html
3. http://msdn.microsoft.com/en-us/library/ff684176%28v=vs.85%29.aspx   http://en.wikipedia.org/wiki/Microsoft_Direct3D
4. http://blogs.msdn.com/b/directx/archive/2009/05/12/2d-drawing-apis-in-windows.aspx
5. http://msdn.microsoft.com/en-us/library/ms536380%28v=vs.85%29.aspx
6. http://blogs.msdn.com/b/e7/archive/2009/04/25/engineering-windows-7-for-graphics-performance.aspx
7. http://msdn.microsoft.com/en-us/library/ff729480%28v=vs.85%29.aspx
8. http://blogs.msdn.com/b/directx/archive/2009/09/29/comparing-direct2d-and-gdi.aspx
9. http://www.youtube.com/watch?v=ay-gqx18UTM
10. http://www.tomshardware.com/reviews/2d-windows-gdi,2539-11.html http://blogs.msdn.com/b/greg_schechter/archive/2006/05/02/588934.aspx
11. http://en.wikipedia.org/wiki/Desktop_Window_Manager
12. http://blogs.msdn.com/b/greg_schechter/archive/2006/03/05/544314.aspx
13. http://blogs.msdn.com/b/greg_schechter/archive/2006/04/02/566767.aspx

Advertisement