Code for Concinnity

beautiful and elegant solutions


Dissecting obscure COM errors

Here’s how I solved these hair-pulling COM errors:

  • 0x800401fd CO_E_OBJNOTCONNECTED Object is not connected to server.
  • 0x8001010e RPC_E_WRONG_THREAD The application called an interface that was marshalled for a different thread.

And this one when I called IGlobalInterfaceTable::GetInterfaceFromGlobal (through CComGITPtr):

  • 0×80070057 E_INVALIDARG The parameter is incorrect.

The basic framework

To make sense of the following code segments, here’s the whole program’s framework to use for the following scenarios:

We use the boost library for threading to make the code less noisy of plumbing work. You don’t really need to grasp them to understand what’s going on.

I have chosen to use the SHDocVw::IShellWindowsPtr interface because it should be present in every Windows environment and simplifies our example without polluting it with the IDL of an example object.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#import "C:/Windows/system32/shdocvw.dll"

#include <iostream>
#include <Windows.h>
#include <comutil.h>

#include <boost/thread.hpp>
#include <boost/ref.hpp>

namespace
{
    // Convenient error checking -- these will fail our
    // program with nicely printed error message in case of
    // any COM error
    void PrintHR(HRESULT hr)
    {
        std::wcerr << L"0x" << std::hex << hr << L" " <<
            _com_error(hr).ErrorMessage() << std::endl;
    }

    void EnsureSuccess(HRESULT hr)
    {
        if(FAILED(hr))
        {
            PrintHR(hr);
            exit(EXIT_FAILURE);
        }
    }

    // A simple wrapper with integrated error checking
    void MyCoInitialize()
    {
        HRESULT hr = CoInitialize(NULL);
        EnsureSuccess(hr);
    }

    // This method can be called in a thread to demonstrate
    // multi-threading issues
    void CreateShellWindow(SHDocVw::IShellWindowsPtr & retval)
    {
        // Because we might be in a different thread, we need
        // to call CoInitialize() just to make sure
        MyCoInitialize();
        HRESULT hr = retval.CreateInstance(__uuiof(SHDocVw::ShellWindows));
        EnsureSuccess(hr);
        CoUninitialize();
    }

    // This should print a numeric number
    // On my machine it always printed "1". The meaning of that
    // is not really important.
    void UseShellWindow(const SHDocVw::IShellWindowsPtr & ptr)
    {
        MyCoInitialize();
        std::cout << ptr->Count << std::endl;
        CoUninitialize();
    }

    void Example()
    {
        // Our example code segments would go here...
    }
}

int main()
{
    try
    {
        Example();
    }
    catch(const _com_error & e)
    {
        PrintHR(e.Error());
        return EXIT_FAILURE;
    }

    return 0;
}

0x800401fd CO_E_OBJNOTCONNECTED Object is not connected to server.

This one is pretty straight forward. You get this when you try to use a COM object after you have called CoUninitialize():

How to reproduce

1
2
3
4
5
6
7
8
9
void Example()
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);
    CoUninitialize();

    UseShellWindow(ptr); // ka-boom!
}

0x8001010e RPC_E_WRONG_THREAD The application called an interface that was marshalled for a different thread.

How to reproduce

1
2
3
4
5
6
7
8
9
10
11
12
void Example()
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;

    CreateShellWindow(ptr);

    // This will use another thread to call `UseShellWindow()`
    boost::thread(&UseShellWindow, ptr); // ka-boom

    CoUninitialize();
}

What happened

The error message is pretty obvious — we need to marshal the pointer for thread A if the pointer was created in thread B, in order to use it in thread A.

How to fix

If you use ATL, one very convenient way to marshal them is to use the GIT (Global Interface Pointer) through CComGITPtr.

(The below serves as a quick fix and introduction. Please look into Global Interface Pointer and stuffs if you want to understand what’s going on).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// A GIT cookie is something you can use to retrieve a pointer
// from the GIT
void UseShellWindowFromCookie(DWORD cookie)
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CComGITPtr<SHDocVw::IShellWindows>(cookie).CopyTo(&fred);

    UseShellWindow(ptr);
    CoUninitialize();
}

void Example()
{
    MyCoInitialize();

    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);

    // This will register our pointer in the GIT and
    // retrieve a GIT cookie in one line
    DWORD cookie = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();

    // Spawn a new thread of the GIT aware version of
    // UseShellWindow()
    boost::thread(&UseShellWindowFromCookie, cookie).join();

    CoUninitialize();
}

0×80070057 E_INVALIDARG The parameter is incorrect.

After discovering how easy it is to marshal COM objects across threads, you may sometimes get E_INVALIDARG when inside IGlobalInterfaceTable::GetInterfaceFromGlobal()

Now this one is really really obscure:

How to reproduce

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
void CreateShellWindowCookie(DWORD & retval)
{
    MyCoInitialize();

    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);

    retval = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();

    CoUninitialize();
}

void Example()
{
    MyCoInitialize();

    // Instead of using a different thread for UseShellWindow()
    // This time around we use a different thread to
    // create the COM pointer
    DWORD cookie;
    boost::thread(&CreateShellWindowCookie, boost::ref(cookie)).join();

    // Let's try to use it from the current thread
    UseShellWindowFromCookie(cookie); // ka-boom!

    CoUninitialize();
}

What happened

This is similar to what happened with CO_E_OBJNOTCONNECTED, only a lot more subtle and obscure this time around.

To understand this, keep this in mind: A COM object is thread aware. A COM object can only ever live in one single thread, even though it can be accessed from another thread (through marshaling).

What that simply means is that when the thread holding the COM object terminates, the COM object becomes invalid. In our example above, CreateShellWindowCookie() was run from a separate thread and finished before we even tried to use it. The actual COM object went down with the thread. Even though we had a cookie, the GIT would fail to find the actual COM object when it tries to do GetInterfaceFromGlobal(). Too bad that it gives such a confusing error message.

To summarize, here are the things to check when you get E_INVALIDARG from GetInterfaceFromGlobal():

  • Maybe the originating thread that called CoCreateInstance() has ended
  • Maybe the originating thread has already called CoUninitialize()
  • Maybe the actual object has already been terminated by someone Releaseing it.
  • Maybe your GIT cookie has already been used. Strangely enough, it seems like the GIT cookie can only be used to get the interface back once.

How to fix

There are 2 basic approaches:

  • Do not create any COM objects from threads other than the main thread (which is guaranteed to last through the program’s life time)
  • Have an elaborate inter-thread communication so that the creator-thread is kept-alive until the COM object’s users are finished with it (This can get really complicated).

To demonstrate, here’s the code that would work:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
void CreateShellWindowCookie(DWORD & retval)
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);

    retval = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();

    // Sleep 10 seconds
    Sleep(10 * 1000);

    CoUninitialize();
}

void Example()
{
    MyCoInitialize();

    // Create a cookie from another thread
    // We do not wait for it to finish
    DWORD cookie;
    boost::thread t(&CreateShellWindowCookie, boost::ref(cookie));

    // We wait 3 seconds
    Sleep(3 * 1000);

    // Now try to use it from cookie
    UseShellWindowFromCookie(cookie); // OK!

    CoUninitialize();
}

There you go

Hopefully this saved you from some hair-pulling experiences :)

Published by kizzx2, on December 4th, 2010 at 3:40 pm. Filled under: Uncategorized Tags: , , , , , • No Comments

Detecting memory leaks with WinDBG the modern (and free) way

This information is surprisingly scarce on the Internet. I should have realized earlier that the world has simply moved on to more modern languages than those who tax us to deal with mundane things like memory leaks.

Anyway, for those who’re still stuck, this should be your savior. This is almost as good as Valgrind on nix

Minimal example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
    #include <crtdbg.h>

    void leak()
    {
        ::operator new(1000);
        new double[1000];
    }

    int main()
    {
        _CrtSetDbgFlag(_CRTDBG_ALLOC_MEM_DF | _CRTDBG_LEAK_CHECK_DF);
        leak();

        return 0;
    }

Running the above code will get you something like this in the Debug output:

Detected memory leaks!
Dumping objects ->
{67} normal block at 0x00573FC8, 8000 bytes long.
 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD 
{66} normal block at 0x005713E8, 1000 bytes long.
 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD 
Object dump complete.

This is nice and good, but it isn’t very useful since we (pretend) don’t know where the leak came from!

And then WinDBG comes in

(Obviously, add WinDBG to your PATH first)

First, we’ll turn on heap stack tracing for our program. gflags.exe needs to be run with elevated privileges:

1
    > gflags /i leak.exe +ust

Now we’re ready to go!

1
    > windbg -g leak.exe

At the debug output in WinDBG, we’ll see our leak report:

Detected memory leaks!
Dumping objects ->
{80} normal block at 0x003B4378, 8000 bytes long.
 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD 
{79} normal block at 0x003B3F50, 1000 bytes long.
 Data: <                > CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD CD 
Object dump complete.

At WinDBG, we can trace down the originating functions

000:0> !heap -p -a 0x003B4378

address 004b4378 found in
_HEAP @ 4b0000
  HEAP_ENTRY Size Prev Flags    UserPtr UserSize - state
    004b4340 03f0 0000  [00]   004b4358    01f64 - (busy)
    Trace: cfba8
    7753dd4d ntdll!RtlAllocateHeap+0x00000274
    6fa87743 MSVCR100D!_heap_alloc_base+0x00000053
    6fa95d8c MSVCR100D!_heap_alloc_dbg_impl+0x000001fc
    6fa95b2f MSVCR100D!_nh_malloc_dbg_impl+0x0000001f
    6fa95adc MSVCR100D!_nh_malloc_dbg+0x0000002c
    6fa9906b MSVCR100D!malloc+0x0000001b
    6fa871b1 MSVCR100D!operator new+0x00000011
    6fa872ef MSVCR100D!operator new[]+0x0000000f


    dc13d5 leak!leak+0x00000035

    dc1447 leak!main+0x00000037
    dc19df leak!__tmainCRTStartup+0x000001bf
    dc180f leak!mainCRTStartup+0x0000000f
    75f43677 kernel32!BaseThreadInitThunk+0x0000000e
    774f9d42 ntdll!__RtlUserThreadStart+0x00000070
    774f9d15 ntdll!_RtlUserThreadStart+0x0000001b

There you go! The highlighted line above is the culprit. For those unfamiliar with WinDBG, here’s a little more hand holding: Click on the source code view, Ctrl+G and then type leak+0x35.

Remember to turn gflags off afterwards. The gflags is a registry and won’t clear even if you delete the file itself.

1
    > gflags /i leak.exe -ust

32/64 bit

To debug 32 bit applications in a 64 bit system, you need to use the 32 bit version of WinDBG, which is not installed by default. You can find dbg_x86_*.msi somewhere in your “Microsoft SDKs” installation directory.

More advanced usage

Heap debugging is not restricted to detecting leaks. If you find your application allocating a lot of memory in some places, you can easily trace down where they come from with WinDBG and the !heap extension. This CodeProject article describes how to do just that in depth.

However, for doing that, I find a more modern and less intrusive way to be using XPerf tool. That combined with WinDBG and you’ve got a debugging power-house. (Resources on how to do it is extremely scarce. If you thought this WinDBG technique is obscure, you ain’t seen nothing :p I may write an post about that some time later when I feel like it.)

Wrong way of doing it

MSDN suggests something like this

1
2
3
4
5
6
    #define _CRTDBG_MAP_ALLOC
    #include <stdlib.h>
    #include <crtdbg.h>

    // At the end of your program
    _CrtDumpMemoryLeaks();

This solution is inadequate for a couple of reasons:

  • It only lists source code location for malloc. What’s the point?
  • Calling _CrtDumpMemoryLeaks() at the end of main() would mistakenly report memory allocated by static objects as leaks.

The only good thing about it is that you don’t need to leave your comfort zone of Visual Studio (that’s probably the entire programmer life for many people, unfortunately).

Some other good resources

Published by kizzx2, on October 9th, 2010 at 12:36 am. Filled under: Useful tips Tags: , , , , • No Comments

Reading a Unicode (UTF16) file in Windows (C++)

I can’t believe it’s so convoluted. This is a step-by-step guide from how I discovered the Elegant Way to read a Unicode file in Windows — it’s only 4 lines!

(The Unicode file I’m referring here is what you get when you save a file as Unicode in Notepad — that’s little Endian UTF16)

For the impatient

1
2
3
4
5
6
7
wstring ReadUTF16(const string & filename)
{
    ifstream file(filename.c_str());
    stringstream ss;
    ss << file.rdbuf() << '\0';
    return wstring((wchar_t *)ss.str().c_str());
}

The problem

To read UTF16, one would expect to use the widechar variants from iostream:

1
2
3
4
5
// Failed attemp
wifstream file("file.utf8.txt");
wstringstream ss;
ss << file.rdbuf();
wstring result = ss.str();

However, the above code will give you bogus text.

The investigation

Why? If you debug it and look at memory, you will see ss.str().c_str() evaluates to something like this:

0x00367738  ff 00 fe 00 60 00 4f 00 7d 00 59 00

The first 4 bytes look strikingly suspicious! 0xfffe is the magic sequence (BOM) for little endian Unicode, yet it’s been broken up by ifstream into two wide-chars (4 bytes).

It turns out that iostream will try very hard to conform to the current codepage — something that’s so obscure about C++ that apparently nobody talks about (from my search). In most cases, the code page will be something Latin. iostream goes the extra mile to break up bytes into widechars, because it assumes all files it reads to be single-byte.

I heard you can actually tell iostream to treat the file as UTF16 instead. There is a setting somewhere probably related to (Click it, look 5 seconds and then come back) codecvt. OK, back? That’s definitely not something you want to touch just to read a damn file.

The solution

It turns to be quite anti-climatic, let’s look at our solution again:

1
2
3
4
5
6
7
wstring ReadUTF16(const string & filename)
{
    ifstream file(filename.c_str());
    stringstream ss;
    ss << file.rdbuf() << '\0';
    return wstring((wchar_t *)ss.str().c_str());
}

You see, basically what the above does is to tell iostream to read the file as a single byte file, but don’t break up the bytes into widechars (so things remain nicely packed) and we’ll do a hardcore conversion ourself. Problem solved.

Caveats

I didn’t bother to trace down where the 0xfffe went after the conversion — it just worked and I was done with it. I also suspect some garbage might be appended at the end of the text stream. Again, Worked For Me. In any case, just do some substring to crop out the parts you don’t want.

Why is it marked “Windows”?

This is marked Windows because the code above is not portable — many *nix platforms have sizeof(wchar_t) == 4. In general, most modern POSIX supports UTF8 natively through plain old char. You can also use the excellent libiconv which converts everything to everything in one function call — making all this fuss irrelevant.

Alternatives

If you deal with UTF8, you might want to check out UTF8-CPP. It probably also supports reading UTF16 but I couldn’t find a quick way to make it work. It’s less heavyweight than ICU — but who can beat 4 lines when you just want to read a darn file?

(UTF8-CPP does support reading UTF-16. Unfortunately it only supports converting it to UTF-8, which is something Windows can’t do conveniently)

Published by kizzx2, on August 3rd, 2010 at 12:15 am. Filled under: Interesting things Tags: , , , , , • 1 Comment

cmd-recycle: Delete files from Windows command line

Download links

I wrote another tool to do just this a while ago, but that one stopped working since I migrated to Windows 7 x64. Microsoft said the SHFileOpearation interface (which the old tool used) had been replaced by the IFileOperation interface. So I figured I would dig up my rusty C# again to update it.

Damn, I can’t believe it took a couple of hours, but the result worked amazingly. This one also supports wildcards:

1
recycle file1 file2 supports-wildcards\*.tmp

I’ve gone the extra mile and set up a proper repository this time around on GitHub. Probably some Windows gurus will mock me left and right with some obscure one-liners or something, lol.

Published by kizzx2, on January 6th, 2010 at 3:30 am. Filled under: Useful tips Tags: , , • No Comments

Starting Windows’ Network and Sharing Center from command prompt

It has always looked not-so-cool to have to type “Sharing” at the start menu or worse yet, navigate through the Control Panel just to start it. Today I finally dug out how to do it from TechNet:

1
control /name Microsoft.NetworkAndSharingCenter

Looks like Microsoft decided to go down the verbose path. Maybe I’ll just stick with the old ways :p

Published by kizzx2, on December 29th, 2009 at 9:21 pm. Filled under: Useful tips Tags: , , , • No Comments

gvim — E303: Unable to open swap file for “[No Name]“, recovery impossible

vim is my editor of choice and I install it in Windows. All is working quite good except one little annoyance — when I open the editor fresh and started editing by going into insert mode, it will give this error message:

1
E303: Unable to open swap file for "[No Name]", recovery impossible

This is usually harmless, as soon as I save the file everything would be great, it’s just that 1 – 2 seconds of pause that doesn’t make me feel good, so today I tried to pin down the problem and here’s how it went:

1
2
3
4
5
6
7
:help E303
(So it's related to the swap file. Let's see where our swap file fails to be created.)
:set directory?
directory=.;c:\tmp;c:\temp
(Oh, those are the default %TEMP% which I moved to another drive! But anyway, the first priority should be the current directory. So let's see where we're now)
:pwd
C:\Windows\system32

Solution

OK it’s pretty obvious now. It probably needs a little bit of manual fixing. Let’s put this in our .vimrc

1
2
3
4
5
" Thanks for an anonymous guest to provide this generic, better solution
set directory=.,$TEMP

" This is line I used at first. The above line is better than this
set directory=.,d:\temp

Great, problem solved!

Published by kizzx2, on November 6th, 2009 at 3:58 am. Filled under: Useful tips Tags: , , , • 11 Comments