Code for Concinnity


MiKTeX + XeTeX “\char_make_active:n {“20}%” problem and the solution!

You need to update all your packages. The latest MiKTeX package manager seems rather broken. If you use Task > Update Wizard, you’ll probably meet “The update wizard could not be found”

The solution is to open the Update program directly (type “Update” in Start Menu or find miktex-update.exe. I used both the normal variant and the “(Admin)” variant. A couple of updates later it’s finally working!

Published by kizzx2, on October 12th, 2011 at 9:28 pm. Filled under: UncategorizedNo Comments

Illustrated Haskell

I launched a new site blog Illustrated Haskell where I experiment with a new approach to explain intermediate Haskell concepts, inspired by Khan Academy‘s style. I am not doing videos (yet?) but the style is intended to be narrative and easy to follow.

Check it out 🙂

Published by kizzx2, on September 24th, 2011 at 6:13 pm. Filled under: UncategorizedNo Comments

In Search of Performance in Haskell

I was doing ITA Software’s “Word Numbers” problem. Now there are various solutions on the Web. Reviewing those solutions tell us it’s a graph/grammar problem. Today we’re going to solve it brute-force.

Now let me give a quick recap of the problem:

A "word number" is defined as

wordNumber 1 = "one" wordNumber 2 = "onetwo" wordNumber 3 = "onetwothree" ... wordNumber 12 = "onetwothreefourfivesixseveneightnineteneleventwelve" ...

Find the 51-billion-th letter of the wordNumber Infinity. Assume that letter is found for wordNumber x, also find the sum of 1 to x

(As someone noted in the StackOverflow comments, I actually misinterpreted the original problem. But that doesn’t affect our purpose here.)

A naive algorithm in C++

The idea seems simple: 2 counters, one for sum of numbers and one for length. We go from wordNumber 1 until the length meets 51000000000. Here is how I said it in C++:

OK, nothing surprising. It takes 45 seconds on my machine to execute (g++ -O3). We will use this as a baseline.

First attempt in Haskell

OK, I lied. It was not my first attempt. My initial version had space leaks which led to the program eating up 1GB RAM in several seconds. Using profiling techniques from Real World Haskell, I identified the space leaks and sprinkled Bang patterns everywhere.

So how well does that run?

Well, I don’t know.

As far as I remember, one version finished in 12 minutes, but I am not sure it is the version posted here. In short, the performance was not bearable.

Now if we were using Perl or Ruby, that might not have been too surprising. But for Haskell this seems something obvious is missing.

The solution

So there I went, I asked for help on StackOverflow and in Haskell-cafe.

There I got help from Bryan O’Sullivan who posted his version. Reiner Pope said that with -fllvm, it finishes in about 1.5 minutes. (OK, he did not exactly say that. He said Haskell version is half as fast as C++.)

But it did not work for me

So I downloaded Bryan’s code (we’ll call it RealWordNumber.hs from now on, sorry bad pun 🙂 and off I went in excitement:

ghc -O2 RealWordNumber.hs RealWordNumber Sum: 1 Segmentation fault/access violation in generated code

Since RealWordNumber.hs uses Int, there is an integer overflow in his code. That’s also exactly the reason why I had to use Int64 in my original version.

(I was using Windows where GHC only has 32-bit)

So I patched his code to use Int64 and off I went. It never finished, I had to kill it.

Huh?

The culprit — 64-bit integers are slow in 32-bit programs

OK the major reason turned out to be this. To confirm, I compiled the C++ version to 32-bit using VC++ (cl /Ox WordNumber.cpp). It took 4 minutes to complete.

So I took RealWordNumber.hs to Arch Linux x64. It turned out that Int in GHC x64 is actually Int64, and the program completed in 5 minutes (ghc -O2).

Since the LLVM backend for GHC 7.0.3 seemed broken at the time I write this, and I did not want to go fixing it immediately, I trusted that it would really turn to 1.5 minutes if I ran it with -fllvm.

Now the fun part — breaking down performance improvements

“So,” I figured, “I just needed to take my original version and put it on Arch x64″. So I tried that, and it was still taking longer than my patience allowed (8+ minutes). The following will explain how I got my version up to the same league with RealWordNumber.hs, step by step.

First of all, I tuned the number down to 510 million (510000000) to avoid having to wait all day doing this.

Here is a base line using 510 million as an input:

RealWordNumber -- 1.94 seconds
K2WordNumber   -- 5.41 seconds

From here we see that if I had waited about 15 minutes, I could have seen my original version finish 😛

Now on to how I got it up to speed:

1. Change divMod to quotRem

I think I heard it from #haskell, people say that quotRem is faster than divMod. So I simply changed all divMod to quotRem. Look at what we’ve got:

K2WordNumber-1 -- 4.48 seconds

OK, that was true. A surprising huge speed up.

2. Change Int64 to Int

Since GHC x64’s Int is actually 64-bit, I changed all Int64 to Int and removed the unnecessary fromIntegral calls.

K2WordNumber-2 -- 3.20 seconds

2 simple changes, we are already getting there 😛

3. Change quotRem to separate quot and rem

Since quotRem returns a tuple of boxed Int64, which may need to be constructed and reconstructed. Max Bolingbroke suggested that using quot and rem separately might help. Here’s the experiment result

K2WordNumber-3 -- 3.13 seconds

It might have been an improvement, but might have just been experimental error. Unfortunately it seems like not much speed up was achieved.

4. Removed redundant parameters

It turned out in my original version, there were several redundant parameters being passed recursively in the function solve (the acc label, and curr). I used ghc -Wall to track them and removed them.

K2WordNumber-4 -- 3.16 seconds

No change. Since GHC figured out enough to warn me, it probably did the right thing and optimized them for me anyway.

5. Simplified parameter passing

The original solve function looked like this

solve :: Int64 -> (Int64, Int64, Int64) -> [Int64] -> (Int64, Int64, Int64)
solve !n !acc@(!sumNum, !sumLen, !curr) (!num:nums)
    | sumLen' >= n = (sumNum', sumLen, num)
    | otherwise = solve n (sumNum', sumLen', num) nums
    where
        sumNum' = sumNum + num
        sumLen' = sumLen + wordLength num

To quote Bryan:

You’re passing parameters in three different ways: a regular Int, a 3-tuple, and a list! Whoa.

He is right. I also noticed that the n parameter does not change and does not need to be passed on. Johan Tibell blogged that GHC could in theory create this closure for us. But I did it manually to see what happens:

solve :: Int -> (Int, Int, Int)
solve n = go 0 0 1
    where
        go !sumNum sumLen i
            | sumLen' >= n = (sumNum', sumLen, i)
            | otherwise = go sumNum' sumLen' (i+1)
            where
                sumNum' = sumNum + i
                sumLen' = sumLen + wordLength i

That resulted in

K2WordNumber-5 -- 2.87 seconds

Good.

6. Using unboxed integer types

I replaced quot and rem with these

    (I# a) // (I# b) = I# (a `quotInt#` b)
    (I# a) % (I# b) = I# (a `remInt#` b)

K2WordNumber-6 — 2.77 seconds

Not as much as I had thought

7. Using Data.Vector.Unboxed instead of Data.Array.Unboxed

K2WordNumber-7 -- 2.50 seconds

  1. Using Data.Vector.Generic.unsafeIndex instead of the (!) operator

K2WordNumber-8 -- 2.20 seconds

Wow, this one was huge.

  1. Move the length vectors inside wordLength‘s closure
wordLength :: Int -> Int
wordLength i = go 0 i
    where
        go !pad !n
            | n < 10         = lenOnes `VG.unsafeIndex` n + pad
            | n < 20         = lenTeens `VG.unsafeIndex` (n-10) + pad
            | n < 100        = go (lenTens `VG.unsafeIndex` (n//10) + pad) (n%10)
            | n < 1000       = go (go (7+pad) (n//100)) (n%100)
            | n < 1000000    = go (go (8+pad) (n//1000)) (n%1000)
            | otherwise      = go (go (7+pad) (n//1000000)) (n%1000000)
        (I# a) // (I# b) = I# (a `quotInt#` b)
        (I# a) % (I# b) = I# (a `remInt#` b)
        lenOnes = VU.fromList [0,3,3,5,4,4,3,5,5,4] -- "", "one","two", ...
        lenTens = VU.fromList [0,3,6,6,5,5,5,7,6,6]
        lenTeens = VU.fromList [3,6,6,8,8,7,7,9,8,8] -- first element is "ten" 3

K2WordNumber-9a — 2.20 seconds

No improvement at all… But if I add bangs before the vectors…

!lenOnes = VU.fromList [0,3,3,5,4,4,3,5,5,4] -- "", "one","two", ...
!lenTens = VU.fromList [0,3,6,6,5,5,5,7,6,6]
!lenTeens = VU.fromList [3,6,6,8,8,7,7,9,8,8] -- first element is "ten" 3

K2WordNumber-9 — 2.00 seconds

WOW!! That was a large speed up. It turns out the reason for the speed up is related to strictness. wordLength is not strict in all lenOnes, lenTens and lenTeens, so they might be garbaged collected (?). Putting bangs on them allow them to stay alive. Since we need to use those arrays very frequently, it was better for them to be around all the time.

Haskell did not allow putting bangs on “global” variables, so I missed it in my original bang sprinkling exercise.

OK, so now we have transformed my original sloppy Haskell program to a “well-performing” Haskell program. Since it performs the same as Bryan O’Sullivan (casually) tuned version, I am going to assume it’s probably as fast as it should (before maybe resorting to serious hacks).

And here is the final version of the code:

Moral of the story

  • Haskell is a very fast functional language. But for tight areas like this, always consider giving plain old C a try — it may just save you days of profiling.
  • 64-bit operations in 32-bit programs are slow (*)
  • quotRem is faster than divMod
  • Earlier I argued in #haskell that “if Integer vs Int matters for your performance, your program is probably written wrong”. Well, there are cases where it matters quite a lot. It turns out conversions are quite costly. (Though I still believe people should use Integer by default)
  • Use the wrapper-worker pattern liberally.
  • If you recurse, make sure every variable actually changes in each recursion. (For most cases this will benefit. If you only recurse several times maybe the closure’s overhead will be larger.)
  • Don’t use a list or tuple just because infinite lists look smart.
  • Data.Vector is a better “general case” container than Data.Array
  • Try to reduce the scope of “global” variables. Put them in a closure so you can bang on it.

  • This may be actually the suckage of Windows’ “WOW64″ (32-bit emulation layer). I have not tried to prove this on 32-bit Linux GHC)

Extra: the fastest indexing data structure — functions!?

I heard from #haskell that “pattern matching in Haskell is less than O(n)”. When I heard it, I didn’t really believe it. I have always thought that N pattern matches means translating into N if-then-else statements for the general case. He said something related to Church encoding but I did not see how it could be applied in the general case to avoid N if-then-else statements.

Just for fun, I tried changing the length vectors into functions:

lenOnes 0 = 0
lenOnes 1 = 3
lenOnes 2 = 3
lenOnes 3 = 5
-- ...
lenTeens 9 = 8

K2WordNumber-10 — 2.00 seconds

Yes, pattern matching is as fast as unsafeIndex. WOW! wren nt thornton from the comments below give a more thorough explanation on this one. It turns out GHC is really darn smart and can figure out fast code given “primitive” definitions.

Published by kizzx2, on August 10th, 2011 at 12:21 pm. Filled under: UncategorizedNo Comments

Eclipse CDT hangs in OS X all the time — the solution

The solution — download 32-bit Eclipse. Works like a charm. Kind of anti-climatic 😛

Published by kizzx2, on July 4th, 2011 at 1:37 am. Filled under: UncategorizedNo Comments

“Pretty-printing” STL vector in Eclipse CDT

It’s surprising to see why the C++ debugging scene in Mac/Linux is so behind Visual Studio. One definite strengths of MSVS is the ability to display STL vectors as arrays in the debug view.

If you’re using Eclipse CDT on Mac, you’re probably stuck with GDB 6.8 which doesn’t come with pretty printing. I tried self building GDB 7.2 but Eclipse CDT 8.0 doesn’t seem to be able to interact well with it (please someone come and correct me if I’m wrong).

So here’s how to display std::vector nicely even if you’re using GDB 6.8:

https://cfc.kizzx2.com/wp-content/uploads/2011/07/eclipse-cdt-debug-vector.png

As you can see, the trick is to convert it to a pointer and to use the array syntax *((int*)nums) @ 3

Enjoy 🙂

Published by kizzx2, on July 3rd, 2011 at 12:37 pm. Filled under: UncategorizedNo Comments

cocos2d-x with Xcode 4 from scratch (without template)

This is mainly some quick notes for myself. Getting cocos2d-x to work on iOS is surprisingly involved. Here we go:

Building cocos2d-x

First grab a copy of cocos2d-x

$ git clone https://github.com/cocos2d/cocos2d-x.git

Now we need to build cocos2d-x. After some pondering about it, I conclude that the project is reasonably disorganized. The easiest way to grab a lib out of the mess is to compile its HelloWorld program:

    $ cd cocos2d-x/HelloWorld/ios
    $ xcodebuild -sdk iphonesimulator-4.3 -configuration Debug
    # ...
    # Now we have the .a file ready, let's find out where it is
    $ find . -iname '*.a'
    ./build/Debug-iphonesimulator/libcocos2d.a

Configuring the Xcode project

Now we need to configure our Xcode project to refer to the libcocos2d.a and the headers.

  1. Create a new Window-based Application
  2. Add these frameworks: OpenGLES.framework, QuartzCore.framework, libxml2.dylib
  3. In Build Settings, define the Preprocessor Macro TARGET_IPHONE_SIMULATOR
  4. Add Header Search Paths: cocos2d-x/cocos2dx/include cocos2d-x/cocos2dx/platform/ios cocos2d-x/cocos2dx/platform cocos2d-x/cocos2dx
  5. Finally Linker Flag -lcocos2d

Then we need to change our default project’s structure a bit. Since we’re creating our own window we will:

  1. Delete MainWindow.xib
  2. In *-Info.plist, remove the “main nib file” entry
  3. Change *AppDelegate.m to *AppDelegate.mm

Let’s write some code!

 
// AppDelegate.mm:
 
    - (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
    {
        static MainApplication sMainAppplication;
        self.window = [[UIWindow alloc] initWithFrame:[[UIScreen mainScreen] bounds]];
        EAGLView * glView = [EAGLView viewWithFrame: [self.window bounds]pixelFormat:kEAGLColorFormatRGBA8 depthFormat:GL_DEPTH_COMPONENT16_OES preserveBackbuffer:NO sharegroup:nil multiSampling:NO numberOfSamples:0];
 
        self.window.rootViewController = [[RootViewController alloc] initWithNibName:nil bundle:nil];
        self.window.rootViewController.wantsFullScreenLayout = YES;
        self.window.rootViewController.view = glView;
 
        [self.window makeKeyAndVisible];
        cocos2d::CCApplication::sharedApplication().run();
 
        // ...
    }
 
// MainApplication.cpp
 
    MainApplication::MainApplication()
    {
    }
 
    MainApplication::~MainApplication()
    {
 
    }
 
    bool MainApplication::initInstance()
    {
        return true;
    }
 
    bool MainApplication::applicationDidFinishLaunching()
    {
        cocos2d::CCDirector * director = cocos2d::CCDirector::sharedDirector();
        director->setOpenGLView(&cocos2d::CCEGLView::sharedOpenGLView());
 
        director->setAnimationInterval(1.0 / 60.0);
        director->runWithScene(MainScene::scene());
 
        return true;
    }
 
    void MainApplication::applicationDidEnterBackground()
    {
 
    }
 
    void MainApplication::applicationWillEnterForeground()
    {
    }

MainScene is your typical cocos2d::CCScene.

There we have it!

Published by kizzx2, on June 18th, 2011 at 1:51 am. Filled under: UncategorizedNo Comments

Good quote from Martin Fowler, worth repeating

“Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” – Martin Fowler

Published by kizzx2, on February 1st, 2011 at 1:44 am. Filled under: UncategorizedNo Comments

Learn PKI in 30 Minutes

Lately I’ve been doing some cryptographic/X.509 things, which was quite new to me. I was almost overwhelmed by the plethora of terminology when I tried to learn it. I’ve consolidated my learning in this short essay:

Teach Yourself PKI in 30 Minutes

It’s written with the intention that someone can quickly get a grasp of the overall picture of these term-heavy X.509/PKI things. It’s written in concise language so as to not isolate the “rest of us” who don’t already grok those acronyms.

If you want a more comprehensive understanding, I can’t recommend the book Understanding PKI enough. It was really helpful and covers everything I needed to know.

There is also RFC 5280 which is the de-facto standard specification on this.

Enjoy! Comments are welcome.

Published by kizzx2, on January 27th, 2011 at 11:41 pm. Filled under: UncategorizedNo Comments

My take on assertions vs. exceptions

There’s been an never ending debate of whether assertions should be converted to exceptions — if an assertion check is useful in Debug build, why remove them in release build? There are some cases where you want to do extensive checking (for e.g. walking over the whole array and detect data corruption) and it’s fine to remove those extensive slow checks during production, but most of the time, the performance difference should really be negligible.

4 kinds of strategies to handle broken assumptions

To fully understand the problem domain, it is useful to clarify the concepts. From my understanding, the strategy to handle broken assumptions can be categorized as follows:

cassert

This is what you get with assert in most languages or Debug.Assert in C#.

- The most defining characteristic of cassert is that it is removed in production.
- In theory this can give you a slight performance gain. I find that it’s not useful most of the time — unless you’re dealing with very low level stuffs.
- If you use `cassert`, you assume that **a violation to your assumption is not going to cause catastrophic consequences**. Personally, I find that is a very bold assumption to make. It is basically trying your luck — after violating the assumption, your program has entered undefined behavior land. Allowing the program to continue could result in data loss or the end of the world.
- This is definitely not suitable for mission critical programs where everything must have clearly defined behavior.

Logger

I heard this idea was introduced in Code Complete. The basic idea is that you still have the checking in release build. If the assumption is broken, you write it to a log and allow the program to continue.

- The reason behind is that there is no programmer to fix the bug when the assumption is broken, so there is no point showing the error message box to the user. The program can do nothing to fix itself, so it may as well continue. The logging bit is there to help post-mortem troubleshooting.
- I personally find that it is futile to log things because it rarely gives you enough information to diagnose the problem. Unless you dump the current stack-trace and local variables in the log, a message like “object should not be null!” is not going to help much — you may as well collect a crash dump. If your program is an input-output program, you may as well ask the client to send the input so you can try to reproduce it in a debugger — where you have such a wealth of information at your disposal.
- Continuing after logging suffers the same problem as `cassert` — your program’s behavior is no longer defined. As such, it should not be used in any reliable or mission critical software.

Terminator

The Terminator basically means calling terminate()/abort() in case of a broken assumption. Alternatively, you can call a elaborately designed termination routine that has been rigorously tested.

- The compelling reason to use this strategy is that it’s closest to 100% well defined behavior. Nothing funny should be able to happen. This is particularly desirable if one wrong instruction in your domain can launch a missile or stop a patient’s life support.
- This is also suitable in unstable environment where a broken precondition could be the result of a full heap corruption.
- The Terminator is really the _correct_ solution in terms of theoretical, academic discussion. However, it is also the most inflexible solution in terms of writing useful software — terminating the program with an error dialog box is not the nicest thing to do in modern consumer software.

Exceptions

Finally we have good ol’ exceptions.

- Stack unwinding and the ability to handle them are the defining features of exceptions.
- Exceptions stops execution of the current routine immediately — this means if you use exceptions in a function, your function is guarded against any undefined behavior (provided you fully defined exceptional cases). The responsibility to maintain integrity of the program is now shifted to the exception handler — the outside `try…catch` block.
- Exceptions also offer a high level of flexibility in handling errors — the actual error handling strategy is implemented by your caller. You could call it some sort of [Inversion of Control](https://secure.wikimedia.org/wikipedia/en/wiki/Inversion_of_control), which is usually a nice thing in terms of architecture.
- One potential security hole remains: stack unwinding. Unwinding the stack would call the destructors on local variables — which can do anything.
- From an extremely rigorous perspective, exceptions aren’t really safe. Your broken assumption could be the result of a full heap corruption, in which case it is really not advisable to allow further code execution in stack unwinding. There is also the fact that your caller could ignore the error in a catch block.

So what does it mean?

It means you should always roll your own because there are different needs for different people.

- A mission critical software must have clearly defined behavior and flexibility in error handling is not something they need. They would probably use a variation of the Terminator pattern.
- A video game would probably benefit from cassert because they need every last bit of performance they can get, and they can reasonably believe that their audience’s platform will not launch a missile. On the other hand, video games should “push their luck” in the face of broken assumptions — the show must go on. If the game can still reasonably continue after a failed assertions, it should. (Exercise to reader: how about the game saving part? Probably the most devastating thing a game can do is to delete or corrupt your game saves.)

The take home message

I would suggest people should always write their own strategy to handle broken assumptions. Personally I like to start with something called ENSURE which implements exception style error handling by default. The good thing about a hand rolled solution is that you can always change your strategy later. For a few places where you’d like extensive checking, you can use cassert explicitly.

Published by kizzx2, on December 5th, 2010 at 10:46 pm. Filled under: UncategorizedNo Comments

Dissecting obscure COM errors

Here’s how I solved these hair-pulling COM errors:

- 0x800401fd `CO_E_OBJNOTCONNECTED` Object is not connected to server.
- 0x8001010e `RPC_E_WRONG_THREAD` The application called an interface that was marshalled for a different thread.

And this one when I called IGlobalInterfaceTable::GetInterfaceFromGlobal (through CComGITPtr):

- 0x80070057 `E_INVALIDARG` The parameter is incorrect.

The basic framework

To make sense of the following code segments, here’s the whole program’s framework to use for the following scenarios:

We use the boost library for threading to make the code less noisy of plumbing work. You don’t really need to grasp them to understand what’s going on.

I have chosen to use the SHDocVw::IShellWindowsPtr interface because it should be present in every Windows environment and simplifies our example without polluting it with the IDL of an example object.

#import "C:/Windows/system32/shdocvw.dll"
 
#include <iostream>
#include <Windows.h>
#include <comutil.h>
 
#include <boost/thread.hpp>
#include <boost/ref.hpp>
 
namespace
{
    // Convenient error checking -- these will fail our
    // program with nicely printed error message in case of
    // any COM error
    void PrintHR(HRESULT hr)
    {
        std::wcerr << L"0x" << std::hex << hr << L" " <<
            _com_error(hr).ErrorMessage() << std::endl;
    }
 
    void EnsureSuccess(HRESULT hr)
    {
        if(FAILED(hr))
        {
            PrintHR(hr);
            exit(EXIT_FAILURE);
        }
    }
 
    // A simple wrapper with integrated error checking
    void MyCoInitialize()
    {
        HRESULT hr = CoInitialize(NULL);
        EnsureSuccess(hr);
    }
 
    // This method can be called in a thread to demonstrate
    // multi-threading issues
    void CreateShellWindow(SHDocVw::IShellWindowsPtr & retval)
    {
        // Because we might be in a different thread, we need
        // to call CoInitialize() just to make sure
        MyCoInitialize();
        HRESULT hr = retval.CreateInstance(__uuiof(SHDocVw::ShellWindows));
        EnsureSuccess(hr);
        CoUninitialize();
    }
 
    // This should print a numeric number
    // On my machine it always printed "1". The meaning of that
    // is not really important.
    void UseShellWindow(const SHDocVw::IShellWindowsPtr & ptr)
    {
        MyCoInitialize();
        std::cout << ptr->Count << std::endl;
        CoUninitialize();
    }
 
    void Example()
    {
        // Our example code segments would go here...
    }
}
 
int main()
{
    try
    {
        Example();
    }
    catch(const _com_error & e)
    {
        PrintHR(e.Error());
        return EXIT_FAILURE;
    }
 
    return 0;
}

0x800401fd CO_E_OBJNOTCONNECTED Object is not connected to server.

This one is pretty straight forward. You get this when you try to use a COM object after you have called CoUninitialize():

How to reproduce

void Example()
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);
    CoUninitialize();
 
    UseShellWindow(ptr); // ka-boom!
}

0x8001010e RPC_E_WRONG_THREAD The application called an interface that was marshalled for a different thread.

How to reproduce

void Example()
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
 
    CreateShellWindow(ptr);
 
    // This will use another thread to call `UseShellWindow()`
    boost::thread(&UseShellWindow, ptr); // ka-boom
 
    CoUninitialize();
}

What happened

The error message is pretty obvious — we need to marshal the pointer for thread A if the pointer was created in thread B, in order to use it in thread A.

How to fix

If you use ATL, one very convenient way to marshal them is to use the GIT (Global Interface Pointer) through CComGITPtr.

(The below serves as a quick fix and introduction. Please look into Global Interface Pointer and stuffs if you want to understand what’s going on).

// A GIT cookie is something you can use to retrieve a pointer
// from the GIT
void UseShellWindowFromCookie(DWORD cookie)
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CComGITPtr<SHDocVw::IShellWindows>(cookie).CopyTo(&fred);
 
    UseShellWindow(ptr);
    CoUninitialize();
}
 
void Example()
{
    MyCoInitialize();
 
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);
 
    // This will register our pointer in the GIT and
    // retrieve a GIT cookie in one line
    DWORD cookie = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();
 
    // Spawn a new thread of the GIT aware version of
    // UseShellWindow()
    boost::thread(&UseShellWindowFromCookie, cookie).join();
 
    CoUninitialize();
}

0x80070057 E_INVALIDARG The parameter is incorrect.

After discovering how easy it is to marshal COM objects across threads, you may sometimes get E_INVALIDARG when inside IGlobalInterfaceTable::GetInterfaceFromGlobal()

Now this one is really really obscure:

How to reproduce

void CreateShellWindowCookie(DWORD & retval)
{
    MyCoInitialize();
 
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);
 
    retval = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();
 
    CoUninitialize();
}
 
void Example()
{
    MyCoInitialize();
 
    // Instead of using a different thread for UseShellWindow()
    // This time around we use a different thread to
    // create the COM pointer
    DWORD cookie;
    boost::thread(&CreateShellWindowCookie, boost::ref(cookie)).join();
 
    // Let's try to use it from the current thread
    UseShellWindowFromCookie(cookie); // ka-boom!
 
    CoUninitialize();
}

What happened

This is similar to what happened with CO_E_OBJNOTCONNECTED, only a lot more subtle and obscure this time around.

To understand this, keep this in mind: A COM object is thread aware. A COM object can only ever live in one single thread, even though it can be accessed from another thread (through marshaling).

What that simply means is that when the thread holding the COM object terminates, the COM object becomes invalid. In our example above, CreateShellWindowCookie() was run from a separate thread and finished before we even tried to use it. The actual COM object went down with the thread. Even though we had a cookie, the GIT would fail to find the actual COM object when it tries to do GetInterfaceFromGlobal(). Too bad that it gives such a confusing error message.

To summarize, here are the things to check when you get E_INVALIDARG from GetInterfaceFromGlobal():

- Maybe the originating thread that called `CoCreateInstance()` has ended
- Maybe the originating thread has already called `CoUninitialize()`
- Maybe the actual object has already been terminated by someone `Release`ing it.
- Maybe your GIT cookie has already been used. Strangely enough, it seems like the GIT cookie can only be used to get the interface back _once_.

How to fix

There are 2 basic approaches:

- Do not create any COM objects from threads other than the main thread (which is guaranteed to last through the program’s life time)
- Have an elaborate inter-thread communication so that the creator-thread is kept-alive until the COM object’s users are finished with it (This can get really complicated).

To demonstrate, here’s the code that would work:

void CreateShellWindowCookie(DWORD & retval)
{
    MyCoInitialize();
    SHDocVw::IShellWindowsPtr ptr;
    CreateShellWindow(ptr);
 
    retval = CComGITPtr<SHDocVw::IShellWindows>(ptr).Detach();
 
    // Sleep 10 seconds
    Sleep(10 * 1000);
 
    CoUninitialize();
}
 
void Example()
{
    MyCoInitialize();
 
    // Create a cookie from another thread
    // We do not wait for it to finish
    DWORD cookie;
    boost::thread t(&CreateShellWindowCookie, boost::ref(cookie));
 
    // We wait 3 seconds
    Sleep(3 * 1000);
 
    // Now try to use it from cookie
    UseShellWindowFromCookie(cookie); // OK!
 
    CoUninitialize();
}

There you go

Hopefully this saved you from some hair-pulling experiences 🙂

Published by kizzx2, on December 4th, 2010 at 3:40 pm. Filled under: UncategorizedNo Comments