PInvokeStackImbalance on using tamper protection

KnuddlbaerKnuddlbaer Posts: 13
edited April 5, 2011 8:28AM in SmartAssembly
Hi!

If i use "I want to add tamper protection", i got an MDA:

PInvokeStackImbalance wurde erkannt.
Message: Ein Aufruf an die PInvoke-Funktion "xyz::–" hat das Gleichgewicht des Stapels gestört. Wahrscheinlich stimmt die verwaltete PInvoke-Signatur nicht mit der nicht verwalteten Zielsignatur überein. Ãœberprüfen Sie, ob die Aufrufkonvention und die Parameter der PInvoke-Signatur mit der nicht verwalteten Zielsignatur übereinstimmen.

Is this an error from MDA or an problem of tamper protection?


VS: 2010 SP1
OS: Windows 7 x64

I debugged an exe that using an smartassembled dll with tamper protection.

Comments

  • Brian DonahueBrian Donahue Posts: 6,590 Bronze 1
    I couldn't say what's going on there. PInvokeStackImbalance happens when the PInvoke signature does not match the signature of the unmanaged DLL. Of all the things SA does, I can't imagine it will change any PInvoke signatures.
  • I was not able to reproduce this problem in an small project.

    I'll write here if i found my error or build an small example.

    Best regards
    Michael
  • Brian DonahueBrian Donahue Posts: 6,590 Bronze 1
    ...or if the arguments that got to the PInvoke call aren't the right length. Maybe string encryption is messing with some of the input variables or something.
  • It happens only if i used tamper protection. I'm new to smartassembly, i don't know what tamperprotection changes.

    This problem will happen if you use an .Net 4.0 project.

    Steps to reproduce:

    Create an ConsoleApplication App and an DLl app. Create an function in the dll and sign it with strong keyname. Protect the dll with tamper protection. Use the class and function in console application.
    Start the project in Debugger with MDA Messages anabled.

    MDA Message:
    http://haispeed.de/redgate/MDA%20Message.jpg

    VS2010 Settings:
    http://haispeed.de/redgate/Settings%20for%20VS2010.jpg
    http://haispeed.de/redgate/Settings%20VS2010%202.jpg

    Testproject:
    http://haispeed.de/redgate/ConsoleApplication1.zip

    If you remove tamper protection, there is no MDA Message.

    Examplemessage without obfuscating:
    Ein Aufruf an die PInvoke-Funktion "Haispeed.Base2!Haispeed.Base2.FrameWorkHelper::StrongNameSignatureVerificationEx"

    Best regards

    Michael Roth
  • Brian DonahueBrian Donahue Posts: 6,590 Bronze 1
    I have many projects using SmartAssembly that have Pinvoke calls and they all work. I am looking at your project. The bit that fails is doing "something" with checking the key so I think you may have a special case where you may not be able to use tamper protection.
  • Brian DonahueBrian Donahue Posts: 6,590 Bronze 1
    Yes, I get the StackImbalance even before using SmartAssembly. All I did was make my own strong-naming key and signed with that. I had even changed the path you see in Program.cs.

    What does this code do, anyway?
  • Let's build the project step by step:

    First, pls create C:\t\redgate_13136 directory so we have the same structure.

    Next, create an empty .Net 4.0 class library project named ClassLibrary1

    Add this function:
        public class Class1
        {
            public void Function()
            {
                Console.WriteLine("Hello World");
            }
        }
    

    and use C:\t\redgate_13136\test.snk to sign the dll.

    Build the dll and copy the result to C:\t\redgate_13136

    Next, Build an ConsoleApplication Project. Add a reference to C:\t\redgate_13136\ClassLibrary1.dll

    Add following code for Main:
        class Program
        {
            static void Main(string[] args)
            {
                Class1 c1 = new Class1();
                c1.Function();
                Console.ReadLine();
            }
        }
    

    Make shure, MDA is enabled:
    http://haispeed.de/redgate/MDA%20Message.jpg

    Run the sample in VS2010. There should no MDA Messages about pInvoke and stack.

    You find this step at http://www.haispeed.de/redgate/13136/Step1.zip

    Next, open smartassembly.

    Create new project. Browse for Assembly C:\t\redgate_13136\ClassLibrary1\ClassLibrary1\bin\Debug\ClassLibrary1.dll

    Set Target to C:\t\redgate_13136\ClassLibrary1.dll

    Change Strong Name Signing to "I want to sign my assembly with a strong name". Browse for an key and select C:\t\redgate_13136\test.snk

    Say: I want to add tamper protection

    http://haispeed.de/redgate/13136/SA_Setting.jpg

    All other otpions leave as is.

    Build the SA project and rebuild (full rebuild) the ConsoleApplication1. Start the app. There should now an MDA Message:

    http://haispeed.de/redgate/13136/MDA-Message.jpg

    Ein Aufruf an die PInvoke-Funktion "ClassLibrary1!ClassLibrary1.Class1::StrongNameSignatureVerificationEx" hat das Gleichgewicht des Stapels gestört. Wahrscheinlich stimmt die verwaltete PInvoke-Signatur nicht mit der nicht verwalteten Zielsignatur überein. Überprüfen Sie, ob die Aufrufkonvention und die Parameter der PInvoke-Signatur mit der nicht verwalteten Zielsignatur übereinstimmen.

    You find the Step on You find this step at http://www.haispeed.de/redgate/13136/Step2.zip


    Version of SmartAssembly is 6.0 buiild 513
  • Brian DonahueBrian Donahue Posts: 6,590 Bronze 1
    Hello,

    I see the problem now, but it looks to only affect the MDA when you debug in Visual Studio. If you run the code outside of VS/disable the PInvoke MDA, there is no problem. I will log this is an annoyance but I'm not sure it's a problem or not.
Sign In or Register to comment.