UScript debug helper library

Share interesting stuff you have found or created yourself.
Post Reply
Chris
Experienced
Posts: 134
Joined: Mon Nov 24, 2014 9:27 am

UScript debug helper library

Post by Chris »

I created this debug helper mainly to make it easier for myself to find script errors (accessed nones and such).
When the binary is hooked, it'll display the name of the variable/function that was accessed through a none, as well as the entire stack leading to the null pointer (casts, variables...).
I thought it might be of use to others, so I'm posting it here, as well as the full C++ source code for it.

Here is an example of how it might look:

Code: Select all

ScriptWarning: TeamSpawnpoint MH-LostSouls.TeamSpawnpoint9 (Function TeamMonster.TeamSpawnpoint.Create:0032) Accessed None 'prototype' Next Op '1' Stack: Try Fetching context >>  , Instance variable: Factory Null 
ScriptWarning: SkaarjWarrior MH-LostSouls.SkaarjWarrior2 (Function UnrealShare.Skaarj.SpinDamageTarget:002D) Accessed None 'Location' Next Op '1' Stack: Try Fetching context >>  , Instance variable: Target Null 
Where the name in the first quoute is the name of the variable or function (including native functions).
The quote after Next Op is the next opcode index to be executed.
The context fetch is the member access operator "." or dot. After that, you'll get a list of all the variable reference / casting opcodes, property name, and whether any of them are null so you know where the issue lies.
An instance variable is an object defined variable, i.e defined in a class scope.
A Local variable is a local function variable, or a parameter.
A default variable is a class default variable (accessed using .default.)

The library is fairly simple. It replaces some of the variable reference opcodes, cast opcodes, context opcodes, assign opcode and so on in the dll.
The UScript class is empty, it's only there as a dummy reference to make the engine load it and bind the dll.
Due to the extra bookkeeping and lookups, they will take slightly longer to execute.

In order to use it, just load it as a mutator or spawn the DebugDummy.
Attachments
UDebug.rar
(868.79 KiB) Downloaded 72 times
User avatar
PrinceOfFunky
Godlike
Posts: 1200
Joined: Mon Aug 31, 2015 10:31 pm

Re: UScript debug helper library

Post by PrinceOfFunky »

Awesome! Any way to get the line number and a opcode table to know what those numbers reference to?
"Your stuff is known to be buggy and unfinished/not properly tested"
User avatar
Barbie
Godlike
Posts: 2792
Joined: Fri Sep 25, 2015 9:01 pm
Location: moved without proper hashing

Re: UScript debug helper library

Post by Barbie »

Because the source code is compiled into byte code and the byte code can be executed although the source code could have been deleted it indicates that there is no connection between source and byte code. But let's see what Chris replies.
"Multiple exclamation marks," he went on, shaking his head, "are a sure sign of a diseased mind." --Terry Pratchett
User avatar
sektor2111
Godlike
Posts: 6403
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: UScript debug helper library

Post by sektor2111 »

I downloaded this and I did not asked to myself such things. Usually if I check my mod(s) and I want to find where the trouble is, definitely I won't remove source-code if mod is not ready... Right ?
I know those "Target" things and "Prototype" problem but... other newer toys which I'm doing might need some debugging...
Greetings Chris !
Chris
Experienced
Posts: 134
Joined: Mon Nov 24, 2014 9:27 am

Re: UScript debug helper library

Post by Chris »

The execution of the compiled code do not rely on the source code at all. They simply made it so that the serializer dumps the source code into the package along with the compiled bytecode. UScript uses UProperty meta objects to briefly describe member types, names and their offsets. This is what allows functions such as GetPropertyText and SetPropertyText to function using the name of a member. The compiler reserves 4 bytes in the byte code right after the 1 byte opcode number for one of the following member types ;
- UBoolProperty
- UIntProperty
- UStrProperty
- UNameProperty
- UFloatProperty
- UByteProperty
- UFixedArrayProperty
- UClassProperty
- UArrayProperty (Dynamic array)
- UMapProperty (Mapped array)
- UStructProperty
- UObjectProperty

The dynamic UScript linker then iterates over these reserved slots and fills them in with the address to the meta objects in runtime, similar to how WIndows and Linux loads dlls and so files. This gives the uscript instruction direct access to the meta object without having to look for it, in order to get the offset for the desired member variable in the given context (local, instance, default). It would look something like this:

Code: Select all

	GProperty = (UProperty*)Stack.ReadObject();
	GPropAddr = Stack.Locals + GProperty->Offset;
GPropAddr would now store the address to the variable. It's messy and slow, but this is how they made it.
Functions are even slower. Any virtual function is being searched for (requires iterations) when called. Only final functions are treated the same way as the above (address stored in the byte code itself).
Virtual functions are poorly hashed using the first 8 bits of the name hash and placed in hash bins that are searched everytime you call a virtual UScript function.

Therefore, it's possible to determine the name of the variables, functions and classes, but not what line the code corresponds to. That would take up a lot of extra, unnecessary space in the byte code, and would be impossible to do without rewriting the compiler.
It would be interesting to rewrite the compiler to generate an external debug database, similar to what native debuggers use to break at certain lines. That would then make it possible to determine the line the code corresponds to.

The opcode printed in the log isn't particularly useful unless you look at, and understand the native part of the Core. The opcode will correspond to an index into the GNatives table.
Epic initially made opcodes occupy 1 byte, this means a maximum of 256 opcodes could be used. This created severe limitations since it would limit the number of native functions + intrinsics to 256.
Therefore they extended it to 2 bytes, a so called High Native. The engine first reads the 1 byte (Stack.Step) like it's in an 8-bit mode, jumps to that index in the GNatives table, and if the index is >= 0x60, that index will point to a native function in the table that reads a second byte, adds them together (sort of like a dispatch that turns the engine into 16-bit mode).
It would then jump to the target native function using the new 16 bit opcode. Yeah the engine is pretty messy and hackish. So for native functions (declared in UScript as native), the printed opcode would point to the opcode number of that function. For opcodes < 0x60, it would correspond to either an internal script instruction, or the few low native functions declared in Object.uc. I put it in there to help me debug the debug library :loool:

The reason it isn't possible to simply open a package in a text editor, remove the source code and then save it again is because the package format contain headers that store offsets and sizes. If you simply erase the source code, without having all of those offsets and sizes changed accordingly, the package becomes corrupt and useless. In order to remove source code, the engine has to load the serialized objects, empty the text buffers, then save the objects again using recalculated offsets and sizes.
User avatar
PrinceOfFunky
Godlike
Posts: 1200
Joined: Mon Aug 31, 2015 10:31 pm

Re: UScript debug helper library

Post by PrinceOfFunky »

Can you make it work with replication?
When a player joins a server with UDebug, it won't print in the client logs.
"Your stuff is known to be buggy and unfinished/not properly tested"
User avatar
Gustavo6046
Godlike
Posts: 1462
Joined: Mon Jun 01, 2015 7:08 pm
Personal rank: Resident Wallaby
Location: Porto Alegre, Brazil
Contact:

Re: UScript debug helper library

Post by Gustavo6046 »

Is it maybe possible to make a parser that maps predicted bytecode positions to line numbers when reading a UScript file, and saves those, so they can be used by UDebug to look the line numbers up later?
"Everyone is an idea man. Everybody thinks they have a revolutionary new game concept that no one else has ever thought of. Having cool ideas will rarely get you anywhere in the games industry. You have to be able to implement your ideas or provide some useful skill. Never join a project whose idea man or leader has no obvious development skills. Never join a project that only has a web designer. You have your own ideas. Focus on them carefully and in small chunks and you will be able to develop cool projects."

Weapon of Destruction
User avatar
PrinceOfFunky
Godlike
Posts: 1200
Joined: Mon Aug 31, 2015 10:31 pm

Re: UScript debug helper library

Post by PrinceOfFunky »

removed_cause_Debug_dll.png
removed_cause_Debug_dll.png (20.96 KiB) Viewed 1458 times
I was playing a single player match with Debug.dll hooked before joining this server, was that the cause?
"Your stuff is known to be buggy and unfinished/not properly tested"
Chris
Experienced
Posts: 134
Joined: Mon Nov 24, 2014 9:27 am

Re: UScript debug helper library

Post by Chris »

Gustavo6046 wrote: Sun Oct 20, 2019 2:52 am Is it maybe possible to make a parser that maps predicted bytecode positions to line numbers when reading a UScript file, and saves those, so they can be used by UDebug to look the line numbers up later?
Pardon my late reply.
This would be possible if the compiler would be rewritten to output an additional debug database of some sort, just like MSVC does it. That database file could then be used during runtime debugging.
PrinceOfFunky wrote: Mon Jan 06, 2020 5:30 pm
removed_cause_Debug_dll.png
I was playing a single player match with Debug.dll hooked before joining this server, was that the cause?
You need to tell ACE to whitelist the mod. Is this your own server?
User avatar
Gustavo6046
Godlike
Posts: 1462
Joined: Mon Jun 01, 2015 7:08 pm
Personal rank: Resident Wallaby
Location: Porto Alegre, Brazil
Contact:

Re: UScript debug helper library

Post by Gustavo6046 »

Database? Just assign each symbol (and instruction) its associated line number (and perhaps column number) in compile time. It is a matter of adding two members to each struct, and knowing what line position you are in during the tokenization and parsing steps. Microsoft stuff tends to boil down to Rube Goldberg, but either way, you get the point.
"Everyone is an idea man. Everybody thinks they have a revolutionary new game concept that no one else has ever thought of. Having cool ideas will rarely get you anywhere in the games industry. You have to be able to implement your ideas or provide some useful skill. Never join a project whose idea man or leader has no obvious development skills. Never join a project that only has a web designer. You have your own ideas. Focus on them carefully and in small chunks and you will be able to develop cool projects."

Weapon of Destruction
a9902957@nepwk.com
Experienced
Posts: 85
Joined: Thu Nov 03, 2011 5:12 am

Re: UScript debug helper library

Post by a9902957@nepwk.com »

Thanks! This might come in handy. Already did. Thanks for this indeed! :gj:
Post Reply