Reversing VAC winapi hooks

Hello there, been a long time ! Today i will talk about the implementation of winapi hooks inside Valve Anti Cheat (aka VAC).

A bit of context

Everything i will say is directly from this topic on UC

What is hooking ?

In the first place we need to properly understand the way hooking works. To hook a function you need to overwrite the first 5 bytes to jmp 0xYOURADDR where 0xYOURADDR is the memory address for the function that will replace the original (a pointer basically). We can also save the original function pointer to return to the original function after our code, this is called a trampoline (tramp) hook.

What's up with VAC ?

Some guy reported that VAC may be hooking VirtualProtect to check if you're messing with the game's code. Another one showed that it has already been reported on UC in the past

Reversing VAC

VAC is actually in GameOverlayRenderer.dll, which manages... the Steam overlay you got it. Pretty dumb design if you ask me but VAC always tries to think outside of the box. From now we just open the dll in IDA, search for the string "VirtualProtect" and we find it ! Here it is after some renaming :

We can see that VAC actually hooks a ton of those winapi functions. For the code, it's pretty straight forward :