Anatomy of a Verifier DLL

A Verifier DLL (Dynamic Link Library) is a special type of DLL used primarily by the Application Verifier tool in Windows to test and verify the behavior of applications, especially in terms of stability, security, and compliance with best practices. The Application Verifier is designed to detect common programming mistakes, such as memory leaks, buffer overflows, and API misuse, which can lead to crashes or security vulnerabilities.

Key Points About Verifier DLLs:

1. Purpose: Verifier DLLs are loaded by the Application Verifier to monitor and validate the behavior of applications at runtime. They check for various issues like memory corruption, resource leaks, and incorrect API usage.
2. Functionality: These DLLs wrap around standard Windows APIs and intercept function calls made by the application. They perform additional checks and validation before allowing the function to proceed, thereby identifying potential issues.
3. Usage: Developers and testers use Verifier DLLs in conjunction with the Application Verifier tool to identify bugs and security issues in their applications, especially during the development and testing phases.
4. Impact on Performance: When Verifier DLLs are enabled, there is an overhead in terms of performance because of the additional checks being performed. This is expected during testing, and Verifier DLLs are typically disabled in production environments.

Practical Explanation

Let’s say you are developing a C++ application, and you want to ensure that it doesn’t have any memory leaks or buffer overflows. You can use the Application Verifier along with its Verifier DLLs to monitor your application.

1. Setup: You enable the Application Verifier for your application. The tool will load the appropriate Verifier DLLs when your application runs.
2. Execution: As your application runs, every time it makes a call to a Windows API (like CreateFile, HeapAlloc, etc.), the Verifier DLL intercepts this call. For example, if your application calls HeapAlloc to allocate memory, the Verifier DLL checks whether this allocation is valid and if any previous allocations have been properly freed.
3. Detection: If the Verifier DLL detects an issue (e.g., a memory leak where allocated memory isn’t freed), it logs the error or may even cause a breakpoint if running under a debugger. This allows you to inspect the issue immediately.
4. Result: After running the application under the Application Verifier, you get detailed logs or immediate notifications about issues like memory corruption, improper API usage, or resource leaks.

Practical Example

Consider a scenario where you have a simple C++ application that allocates memory but forgets to free it:

#include <windows.h>

int main() {
    int* p = (int*)HeapAlloc(GetProcessHeap(), 0, sizeof(int) * 100);
    // Memory allocated, but not freed
    return 0;
}

If you run this application with the Application Verifier enabled, the Verifier DLL will detect that the memory allocated by HeapAlloc was not freed before the application terminated. It will report this as a memory leak, allowing you to correct the issue by adding the corresponding HeapFree call.

Creating a Sample Verifier DLL

Creating a Verifier DLL from scratch is a complex and advanced task that involves deep knowledge of Windows internals, API hooking, and possibly some undocumented behavior. However, I can guide you through a simplified example that conceptually illustrates how a Verifier DLL might work. This example will cover several Windows operations and demonstrate how you could theoretically monitor and validate them.

Step-by-Step Overview:

1. Create a DLL Project: We’ll create a basic DLL in C++ that hooks into specific Windows API calls.
2. Hook Functions: We’ll set up hooks for some common Windows API functions.
3. Validate Operations: Implement validation logic to detect incorrect usage of these functions.
4. Log or Report Issues: If an issue is detected, we’ll log the error or trigger a debugger breakpoint.

Target Operations:

1. HeapAlloc: Monitor memory allocation and ensure it’s followed by HeapFree.
2. CreateFile: Check if files are opened without being closed.
3. RegOpenKeyEx: Validate registry keys being opened.
4. VirtualAlloc: Monitor virtual memory allocation.
5. CloseHandle: Check if handles are properly closed.

Simplified Code Example

This code will focus on monitoring HeapAlloc and HeapFree. It will track allocations and ensure they are freed before the program terminates.

#include <windows.h>
#include <detours.h>
#include <map>

#pragma comment(lib, "detours.lib")

typedef LPVOID (WINAPI* pHeapAlloc)(HANDLE, DWORD, SIZE_T);
typedef BOOL (WINAPI* pHeapFree)(HANDLE, DWORD, LPVOID);

pHeapAlloc OriginalHeapAlloc = nullptr;
pHeapFree OriginalHeapFree = nullptr;

// A map to keep track of allocated memory
std::map<LPVOID, SIZE_T> g_Allocations;

LPVOID WINAPI HookedHeapAlloc(HANDLE hHeap, DWORD dwFlags, SIZE_T dwBytes) {
    LPVOID pMem = OriginalHeapAlloc(hHeap, dwFlags, dwBytes);
    if (pMem) {
        g_Allocations[pMem] = dwBytes;
        OutputDebugString(L"Memory allocated");
    }
    return pMem;
}

BOOL WINAPI HookedHeapFree(HANDLE hHeap, DWORD dwFlags, LPVOID lpMem) {
    if (g_Allocations.find(lpMem) != g_Allocations.end()) {
        g_Allocations.erase(lpMem);
        OutputDebugString(L"Memory freed");
    }
    return OriginalHeapFree(hHeap, dwFlags, lpMem);
}

void VerifyAllocations() {
    if (!g_Allocations.empty()) {
        OutputDebugString(L"Memory leaks detected:");
        for (auto& alloc : g_Allocations) {
            wchar_t buffer[256];
            swprintf(buffer, 256, L"Leaked memory at address %p, size: %d bytes", alloc.first, (int)alloc.second);
            OutputDebugString(buffer);
        }
    } else {
        OutputDebugString(L"No memory leaks detected.");
    }
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved) {
    switch (ul_reason_for_call) {
    case DLL_PROCESS_ATTACH:
        // Hook the functions
        OriginalHeapAlloc = (pHeapAlloc)DetourFunction((PBYTE)HeapAlloc, (PBYTE)HookedHeapAlloc);
        OriginalHeapFree = (pHeapFree)DetourFunction((PBYTE)HeapFree, (PBYTE)HookedHeapFree);
        break;
    case DLL_PROCESS_DETACH:
        // Unhook the functions and verify allocations
        DetourRemove((PBYTE)OriginalHeapAlloc, (PBYTE)HookedHeapAlloc);
        DetourRemove((PBYTE)OriginalHeapFree, (PBYTE)HookedHeapFree);
        VerifyAllocations();
        break;
    }
    return TRUE;
}

Explanation of the Code:

1. Function Typedefs: We define typedefs for the HeapAlloc and HeapFree functions. This allows us to store pointers to the original functions before we hook them.
2. Hooked Functions:HookedHeapAlloc: This function replaces the original HeapAlloc and records the memory allocation in a map.
   HookedHeapFree: This function replaces the original HeapFree and removes the corresponding memory allocation from the map when the memory is freed.
3. VerifyAllocations: When the DLL is unloaded (on process detachment), we check if there are any remaining allocations that weren’t freed, indicating a memory leak. This is logged to the debug output.
4. Detour Setup: DetourFunction from the Detours library is used to redirect calls from HeapAlloc and HeapFree to our hooked versions. When the DLL is unloaded, we remove the hooks.

Extending to Other Operations:
You can extend this example to other operations like CreateFile, RegOpenKeyEx, VirtualAlloc, etc., by similarly hooking these functions and implementing the corresponding validation logic.

Conclusion

This simplified example shows how a Verifier DLL can be created to monitor specific operations in a Windows application. While the example focuses on memory allocation, the same principles can be applied to other system resources. Creating a fully-fledged Verifier DLL involves much more, including robust error handling, performance considerations, and comprehensive coverage of the API set being monitored.

#WindowsInternals #DLL #VerifierDLL #OperatingSystem