PInvokeStackImbalance on using tamper protection

Knuddlbaer

Hi!

If i use "I want to add tamper protection", i got an MDA:

PInvokeStackImbalance wurde erkannt.
Message: Ein Aufruf an die PInvoke-Funktion "xyz::–" hat das Gleichgewicht des Stapels gestört. Wahrscheinlich stimmt die verwaltete PInvoke-Signatur nicht mit der nicht verwalteten Zielsignatur überein. Ãœberprüfen Sie, ob die Aufrufkonvention und die Parameter der PInvoke-Signatur mit der nicht verwalteten Zielsignatur übereinstimmen.

Is this an error from MDA or an problem of tamper protection?


VS: 2010 SP1
OS: Windows 7 x64

I debugged an exe that using an smartassembled dll with tamper protection.

Brian Donahue

I couldn't say what's going on there. PInvokeStackImbalance happens when the PInvoke signature does not match the signature of the unmanaged DLL. Of all the things SA does, I can't imagine it will change any PInvoke signatures.

Knuddlbaer

I was not able to reproduce this problem in an small project.

I'll write here if i found my error or build an small example.

Best regards
Michael

Brian Donahue

...or if the arguments that got to the PInvoke call aren't the right length. Maybe string encryption is messing with some of the input variables or something.

Knuddlbaer

It happens only if i used tamper protection. I'm new to smartassembly, i don't know what tamperprotection changes.

This problem will happen if you use an .Net 4.0 project.

Steps to reproduce:

Create an ConsoleApplication App and an DLl app. Create an function in the dll and sign it with strong keyname. Protect the dll with tamper protection. Use the class and function in console application.
Start the project in Debugger with MDA Messages anabled.

MDA Message:
http://haispeed.de/redgate/MDA%20Message.jpg

VS2010 Settings:
http://haispeed.de/redgate/Settings%20for%20VS2010.jpg
http://haispeed.de/redgate/Settings%20VS2010%202.jpg

Testproject:
http://haispeed.de/redgate/ConsoleApplication1.zip

If you remove tamper protection, there is no MDA Message.

Examplemessage without obfuscating:
Ein Aufruf an die PInvoke-Funktion "Haispeed.Base2!Haispeed.Base2.FrameWorkHelper::StrongNameSignatureVerificationEx"

Best regards

Michael Roth

Brian Donahue

I have many projects using SmartAssembly that have Pinvoke calls and they all work. I am looking at your project. The bit that fails is doing "something" with checking the key so I think you may have a special case where you may not be able to use tamper protection.

Brian Donahue

Yes, I get the StackImbalance even before using SmartAssembly. All I did was make my own strong-naming key and signed with that. I had even changed the path you see in Program.cs.

What does this code do, anyway?

Knuddlbaer

Let's build the project step by step:

First, pls create C:\t\redgate_13136 directory so we have the same structure.

Next, create an empty .Net 4.0 class library project named ClassLibrary1

Add this function:

    public class Class1
    {
        public void Function()
        {
            Console.WriteLine("Hello World");
        }
    }

and use C:\t\redgate_13136\test.snk to sign the dll.

Build the dll and copy the result to C:\t\redgate_13136

Next, Build an ConsoleApplication Project. Add a reference to C:\t\redgate_13136\ClassLibrary1.dll

Add following code for Main:

    class Program
    {
        static void Main(string[] args)
        {
            Class1 c1 = new Class1();
            c1.Function();
            Console.ReadLine();
        }
    }

Make shure, MDA is enabled:
http://haispeed.de/redgate/MDA%20Message.jpg

Run the sample in VS2010. There should no MDA Messages about pInvoke and stack.

You find this step at http://www.haispeed.de/redgate/13136/Step1.zip

Next, open smartassembly.

Create new project. Browse for Assembly C:\t\redgate_13136\ClassLibrary1\ClassLibrary1\bin\Debug\ClassLibrary1.dll

Set Target to C:\t\redgate_13136\ClassLibrary1.dll

Change Strong Name Signing to "I want to sign my assembly with a strong name". Browse for an key and select C:\t\redgate_13136\test.snk

Say: I want to add tamper protection

http://haispeed.de/redgate/13136/SA_Setting.jpg

All other otpions leave as is.

Build the SA project and rebuild (full rebuild) the ConsoleApplication1. Start the app. There should now an MDA Message:

http://haispeed.de/redgate/13136/MDA-Message.jpg

Ein Aufruf an die PInvoke-Funktion "ClassLibrary1!ClassLibrary1.Class1::StrongNameSignatureVerificationEx" hat das Gleichgewicht des Stapels gestört. Wahrscheinlich stimmt die verwaltete PInvoke-Signatur nicht mit der nicht verwalteten Zielsignatur überein. Überprüfen Sie, ob die Aufrufkonvention und die Parameter der PInvoke-Signatur mit der nicht verwalteten Zielsignatur übereinstimmen.

You find the Step on You find this step at http://www.haispeed.de/redgate/13136/Step2.zip


Version of SmartAssembly is 6.0 buiild 513

Brian Donahue

Hello,

I see the problem now, but it looks to only affect the MDA when you debug in Visual Studio. If you run the code outside of VS/disable the PInvoke MDA, there is no problem. I will log this is an annoyance but I'm not sure it's a problem or not.