UScript debug helper library

If you have found new Downloads or created something by your own, please announce it here
Post Reply
Chris
Experienced
Posts: 122
Joined: Mon Nov 24, 2014 9:27 am

UScript debug helper library

Post by Chris » Thu Oct 10, 2019 9:31 pm

I created this debug helper mainly to make it easier for myself to find script errors (accessed nones and such).
When the binary is hooked, it'll display the name of the variable/function that was accessed through a none, as well as the entire stack leading to the null pointer (casts, variables...).
I thought it might be of use to others, so I'm posting it here, as well as the full C++ source code for it.

Here is an example of how it might look:

Code: Select all

ScriptWarning: TeamSpawnpoint MH-LostSouls.TeamSpawnpoint9 (Function TeamMonster.TeamSpawnpoint.Create:0032) Accessed None 'prototype' Next Op '1' Stack: Try Fetching context >>  , Instance variable: Factory Null 
ScriptWarning: SkaarjWarrior MH-LostSouls.SkaarjWarrior2 (Function UnrealShare.Skaarj.SpinDamageTarget:002D) Accessed None 'Location' Next Op '1' Stack: Try Fetching context >>  , Instance variable: Target Null 
Where the name in the first quoute is the name of the variable or function (including native functions).
The quote after Next Op is the next opcode index to be executed.
The context fetch is the member access operator "." or dot. After that, you'll get a list of all the variable reference / casting opcodes, property name, and whether any of them are null so you know where the issue lies.
An instance variable is an object defined variable, i.e defined in a class scope.
A Local variable is a local function variable, or a parameter.
A default variable is a class default variable (accessed using .default.)

The library is fairly simple. It replaces some of the variable reference opcodes, cast opcodes, context opcodes, assign opcode and so on in the dll.
The UScript class is empty, it's only there as a dummy reference to make the engine load it and bind the dll.
Due to the extra bookkeeping and lookups, they will take slightly longer to execute.

In order to use it, just load it as a mutator or spawn the DebugDummy.
Attachments
UDebug.rar
(868.79 KiB) Downloaded 9 times

User avatar
PrinceOfFunky
Godlike
Posts: 1077
Joined: Mon Aug 31, 2015 10:31 pm

Re: UScript debug helper library

Post by PrinceOfFunky » Thu Oct 10, 2019 11:40 pm

Awesome! Any way to get the line number and a opcode table to know what those numbers reference to?
"Your stuff is known to be buggy and unfinished/not properly tested"

User avatar
Barbie
Godlike
Posts: 1722
Joined: Fri Sep 25, 2015 9:01 pm
Location: moved without proper hashing

Re: UScript debug helper library

Post by Barbie » Fri Oct 11, 2019 3:32 pm

Because the source code is compiled into byte code and the byte code can be executed although the source code could have been deleted it indicates that there is no connection between source and byte code. But let's see what Chris replies.
"Multiple exclamation marks," he went on, shaking his head, "are a sure sign of a diseased mind." --Terry Pratchett

User avatar
sektor2111
Godlike
Posts: 4135
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: UScript debug helper library

Post by sektor2111 » Fri Oct 11, 2019 5:10 pm

I downloaded this and I did not asked to myself such things. Usually if I check my mod(s) and I want to find where the trouble is, definitely I won't remove source-code if mod is not ready... Right ?
I know those "Target" things and "Prototype" problem but... other newer toys which I'm doing might need some debugging...
Greetings Chris !

Chris
Experienced
Posts: 122
Joined: Mon Nov 24, 2014 9:27 am

Re: UScript debug helper library

Post by Chris » Fri Oct 11, 2019 10:16 pm

The execution of the compiled code do not rely on the source code at all. They simply made it so that the serializer dumps the source code into the package along with the compiled bytecode. UScript uses UProperty meta objects to briefly describe member types, names and their offsets. This is what allows functions such as GetPropertyText and SetPropertyText to function using the name of a member. The compiler reserves 4 bytes in the byte code right after the 1 byte opcode number for one of the following member types ;
- UBoolProperty
- UIntProperty
- UStrProperty
- UNameProperty
- UFloatProperty
- UByteProperty
- UFixedArrayProperty
- UClassProperty
- UArrayProperty (Dynamic array)
- UMapProperty (Mapped array)
- UStructProperty
- UObjectProperty

The dynamic UScript linker then iterates over these reserved slots and fills them in with the address to the meta objects in runtime, similar to how WIndows and Linux loads dlls and so files. This gives the uscript instruction direct access to the meta object without having to look for it, in order to get the offset for the desired member variable in the given context (local, instance, default). It would look something like this:

Code: Select all

	GProperty = (UProperty*)Stack.ReadObject();
	GPropAddr = Stack.Locals + GProperty->Offset;
GPropAddr would now store the address to the variable. It's messy and slow, but this is how they made it.
Functions are even slower. Any virtual function is being searched for (requires iterations) when called. Only final functions are treated the same way as the above (address stored in the byte code itself).
Virtual functions are poorly hashed using the first 8 bits of the name hash and placed in hash bins that are searched everytime you call a virtual UScript function.

Therefore, it's possible to determine the name of the variables, functions and classes, but not what line the code corresponds to. That would take up a lot of extra, unnecessary space in the byte code, and would be impossible to do without rewriting the compiler.
It would be interesting to rewrite the compiler to generate an external debug database, similar to what native debuggers use to break at certain lines. That would then make it possible to determine the line the code corresponds to.

The opcode printed in the log isn't particularly useful unless you look at, and understand the native part of the Core. The opcode will correspond to an index into the GNatives table.
Epic initially made opcodes occupy 1 byte, this means a maximum of 256 opcodes could be used. This created severe limitations since it would limit the number of native functions + intrinsics to 256.
Therefore they extended it to 2 bytes, a so called High Native. The engine first reads the 1 byte (Stack.Step) like it's in an 8-bit mode, jumps to that index in the GNatives table, and if the index is >= 0x60, that index will point to a native function in the table that reads a second byte, adds them together (sort of like a dispatch that turns the engine into 16-bit mode).
It would then jump to the target native function using the new 16 bit opcode. Yeah the engine is pretty messy and hackish. So for native functions (declared in UScript as native), the printed opcode would point to the opcode number of that function. For opcodes < 0x60, it would correspond to either an internal script instruction, or the few low native functions declared in Object.uc. I put it in there to help me debug the debug library :loool:

The reason it isn't possible to simply open a package in a text editor, remove the source code and then save it again is because the package format contain headers that store offsets and sizes. If you simply erase the source code, without having all of those offsets and sizes changed accordingly, the package becomes corrupt and useless. In order to remove source code, the engine has to load the serialized objects, empty the text buffers, then save the objects again using recalculated offsets and sizes.

User avatar
PrinceOfFunky
Godlike
Posts: 1077
Joined: Mon Aug 31, 2015 10:31 pm

Re: UScript debug helper library

Post by PrinceOfFunky » Sat Oct 19, 2019 3:54 pm

Can you make it work with replication?
When a player joins a server with UDebug, it won't print in the client logs.
"Your stuff is known to be buggy and unfinished/not properly tested"

User avatar
Gustavo6046
Inhuman
Posts: 821
Joined: Mon Jun 01, 2015 7:08 pm
Personal rank: Finite
Contact:

Re: UScript debug helper library

Post by Gustavo6046 » Sun Oct 20, 2019 2:52 am

Is it maybe possible to make a parser that maps predicted bytecode positions to line numbers when reading a UScript file, and saves those, so they can be used by UDebug to look the line numbers up later?

Post Reply