Ingame package obfuscator.

[Higor]

This is entirely aimed at developers with basic knowledge about obfuscating packages.
Say you want to create a package with a certain protected function or variables, and you don't want to nuke the entire code or anything.

This thing is a native class that runs on the game.
Since I don't know much about licensing, I'll avoid posting the code for now.
The way it operates is kinda limited and prone to failures, i will explain below.

//Coding to make obfuscation possible
Any name in the name table (variables, functions, classes) can be replaced with gibberish with no problems as long as no external package is intended to read those without hacking.
Any PRIVATE VARIABLE name can be zero'd (no name, no possibility to access from external UScript) as long as all variables with this name are PRIVATE, and no functions or classes use said name.
Create a couple of unused variable names if you're going to do name zeroing (var int bogus; is a good example).
Source code can be partially blanked, even making unrealed unable to load the class's source by using OBFSTART and OBFEND statements inside the class, YOU MUST HAVE BOTH OPENING AND CLOSING STATEMENTS IN THE SAME CLASS.

//Launching the obfuscator
I didn't even bother with an interface or an application, so i used the game's own property window as interface.
Place all files on the \System\ folder.
Launch the game, windowed mode.
Since native packages fail to bind if loaded with inproper letter case, spawn the actor by using the "exec obfuscate" batch, it's just an extensionless text file that does this: summon Obfuscator.PackageObfuscator, editactor class=PackageObfuscator.
The obfuscator actor will be spawned and a window to access it's operating properties will show up immediately.

//How it works.
The obfuscator isn't smart, so you MUST read this carefully:
- Names are case sensitive, code all the vars, functions with same name in the package using the same Upper/Lower case combos. (Test,Test,Test is good) (Test,test,Test is bad).
- The name table will never vary in size, so won't the package. If you zero'd a name, you must increase the length of a different name, specify bogus names for that.
- Names can have up to 64 chars (63 to be safe), so count the chars of every name you remove, for every 40 chars you remove, add a new bogus name to increase length.
Old name table [removeme ;obfuscate ;bogus] > new name table [ ;.qtÆæ¤◄ܺ ;bogusaaaaaaaa], as you can see, the name table doesn't vary in size, but when removing a name, it increases the length in another one.
- Never add non-existant names, the obfuscator doesn't check if those aren't in the package!
- All the data between OBFSTART and OBFEND (add those words in the script) will be zero'd.
- If you don't add an OBFEND after you add OBFSTART in the same class, the obfuscator will delete critical package data until it finds another OBFEND somewhere!, always add OBFEND in the same class after OBFSTART.


//Setting up the obfuscator
Open the [+] PackageObfuscator section on the property window.
Input package is the original package, Output is your edited package. Remember to keep the extension.
Setting bSave to TRUE will save the config, it will go back to FALSE immediately.
Setting bStartNow to TRUE will produce an obfuscated package.
The string arrays (Bogus,Delete,Obfuscate) don't need to be compacted, they work anyways.
When done, try to summon an actor from the newly created package (or load it on UnrealEd), if you get no crash then the package was properly baked

//OBFSTART
When the obfuscator passes thru this thread, this line will cease to exist.
//OBFEND


Download:

Obfuscator.zip

(10.25 KiB) Downloaded 501 times

File list:
Obfuscator.dll
Obfuscator.u
obfuscate
SampleObfuscatedClass.uc
Obfuscator.ini

[JackGriffin]

I see what you did there.

Now pump out the reverse and give me something that reads the hex and spits me out unobfuscated classes

This is an interesting idea and I see why it appeals to the intellectual side of your mind but any new efforts along this line only hurt UScript coding in the long run. No definitive tutorial still exists for coders and the only true way to learn is from the example of others. If someone like Ferali used this it would devastate any later developer's gaining of knowledge.

I've been tempted hard in the last couple of years to post source code for protected mods just because it's dwindled down to so few developers and players overall. I'm encountering this very thing in Unreal coop play right now with an active coder's stripping of his mod work.

[MrLoathsome]

Now pump out the reverse and give me something that reads the hex and spits me out unobfuscated classes

Now that would be an evil program right there. And a neat trick.

Very impressive concept. But the only application I could see for it now in the UT world would be for
new versions of ACE/Pure or stuff like that.

Hell, everything I release contains the freaking Classes folder with all the uc files ready to read or compile.

Which is I suppose the opposite of obfuscation.....

Still very nice work, and would be good if used with new Anti-cheat type stuff.

[Higor]

Jack, a full obfuscator was hanging around for years already.
This thing is a partial obfuscator, obfuscating an entire package with this will be VERY tedious.

The main usage of this is to disable variables from being read from UScript (pure's anti speed hack is an example) and to make sure some functions don't get called so easily.
SiegeUltimate is being obfuscated like this: 4 names gone, 8 names altered, 4 classes with partial or no source( roughly 300 lines of code), mainly on the improved pseudo ZP I made for Siege's built in weapons.
If you wonder how partial is the obfuscation... well, SiegeUltimate has 283 classes and excepting the 4 I mentioned previously, all the others are visible and recompilable.

EDIT:
Theoretically doing the opposite shouldn't be difficult.
It'd still require some more analysis to prevent name collisions, and defining like 50 bogus names that don't belong to imports but rather exports.

EDIT2:
Once you understand how Serialization works (takes some research from working just with the public 432 headers), it's stupidly easy to do and the possibilities are endless.
This is the beginning of the function.

Code: Select all

void APackageObfuscator::execCreatePackage(FFrame &Stack, RESULT_DECL)
{
	guard(APackageObfuscator::execCreatePackage);
	P_FINISH;

	FArchive* Src = GFileManager->CreateFileReader( *InputPackage );
	FArchive* Dest = GFileManager->CreateFileWriter( *OutputPackage, (true?0:FILEWRITE_NoReplaceExisting) | (true?FILEWRITE_EvenIfReadOnly:0) );

	if ( Src && Dest )
	{
		BYTE Buffer[8192]; //To be used with OBFSTART and OBFEND mass parsing (faster to read a buffer than single bytes)
		INT Size = Src->TotalSize();
		BYTE EmptyName[5] = { 0, 0, 0, 7, 0}; //This is our empty name entry

		//Copy the header and record important variables
		Src->Serialize( &Buffer, 12 );
		Dest->Serialize( &Buffer, 12 );
		
		INT NameCount;
		Src->Serialize( &NameCount, 4);
		Dest->Serialize( &NameCount, 4);
		
		INT NameOffset;
		Src->Serialize( &NameOffset, 4);
		Dest->Serialize( &NameOffset, 4);

		//Our position is 20, serialize up to the beginning of the name table
		Src->Serialize( &Buffer, NameOffset - 20);
		Dest->Serialize( &Buffer, NameOffset - 20);
//To be continued

[Feralidragon]

Rest assured, I won't ever use a obfuscator for anything mine. My code is always open source and free to be re-used.
However, I must point out that obfuscation is downright useless if you aim for security. It may still be useful to protect your code from theft, but other than that relying of obfuscation for security is flawed by design.

As for the obfuscation itself: afaik you can do the obfuscation directly in the compiled package without bogus names defined previously. The name table has a defined length and very rigid rules in its reading (it starts with the table offset, then it just knows the number of names and where they start and end with control bytes), so theoretically as long as you keep the bytes after the original table in the in the exact places, there shouldn't be an issue. So therefore you could end the name table sooner in the file, and fill up the rest with null chars or other garbage until the byte where the table was supposed to end before, without having to define any extra bogus names in the class itself.

And yes, it would easily reversible, too easy in fact. But again, if this is meant just to keep noobs from stealing code, it's a sufficient approach, so I am not worried with someone misusing it. If the case becomes too extreme on misuse with further development, I can eventually write a deobfuscator myself. If the engine can still read it, it's because the basic structure is still there and you can still get everything from the package anyway through the export table.
But in the case of someone relying on this for actual security, obfuscation won't help on anything at all, and I don't know why some people still think it does. The only thing keeping people from hacking packages in UT these days is ACE given that it compares the MD5 hash of packages to ensure they were not hacked, among other things afaik.

Just my 2 cents

[Hook]

I agree with Feralidragon and JackGriffin.
I used to code for PCs, using hex machine language, Basic, and I studied Cobol in college courses and could use it.
I was pretty darn good where I could produce games, graphics programs and apps, with clean, lean efficient code.
That was many years ago, at about the time Apple was just getting started.
I have forgotten most of it now.
I have only looked at UScript and I mostly understand it.
I would LOVE to learn it now, sometime soon I hope, so looking at the coding projects will be mostly what I will learn it with.
Thanks to those who offer the source code with their projects.

[Rakiayn]

One learns on the forums from people who spend their free time helping you and by looking at projects of others. It is childish to hide your code like that.

[JackGriffin]

I totally understand Higor's interest in this and he's done fantastic work with the entire Ucommunity. I don't think his intent is to cause any harm at all, this is more an intellectual exercise. I just shy away from releasing anything that obfuscates code now simply because so few people are still active and almost no new people are coming on board now. Let's take care not to paint him as just "someone enabling people to hide code" because he rightly deserves more respect than that would appear to give him. There are countless threads here where he's helping, offering code, etc. Further although I disagree with hiding any code I do understand his reasoning behind his choices on Seige and why he pursued this avenue. Given the choices facing him it's an acceptable solution to a complex problem. Really we don't part ways until it comes to posting the source code and enabling others to do this so easily. Now that part I wish he would rethink but as is said "You can't put toothpaste back into the tube". Instead I hope that we can encourage the sharing of code and properly deal with those who don't (KAAL), thereby encouraging people not to use things like this.

That being said I don't think most coders looking at this will ever realize the amount of information Higor is presenting for you to learn. Once you understand the machine code you realize how pointless obfuscation becomes really, as Ferali eloquently stated above.

[Higor]

Continued from execCreatePackage:
We create 3 memory blocks (FCharObf128) that contain the names we previously defined, ordered and in single byte chars instead of FString properties, ParseStrings takes care of that.
Zeroing the memory these blocks was necessary here because said blocks don't really initialize themselves.
Unreal Tournament and Unreal could use an addon with the ability to do serialization and other memory related functions from unrealscript, although the UScript -> C++ methods don't really favor the pointer parts (unless we specify OUT parameters in said functions?)

Code: Select all

//Continued
		//Generate our temporary name tables
		FCharObf128 ToDelete, ToReplace, ToBogus;
		appMemzero( &ToDelete, 69*128 + 1); //Same as the ones below, this is for clarity
		appMemzero( &ToBogus, 8833);
		appMemzero( &ToReplace, 8833);
		ParseStrings( &ToDelete, &ToReplace, &ToBogus);

		INT CurNames = 0; //We will use this counter to check names one by one
		INT CharactersToDelete = CharsToDelete(); //We must delete this amount of chars!!
//To be continued

Attaching the other member funcs and structs.
Count chars in a string array, this is very simple to do, even in UnrealScript

Code: Select all

INT APackageObfuscator::CharsToDelete()
{
	INT result = 0;
	
	for ( int i=0; i<128 ; i++ )
	{
		if ( DeleteNames[i].Len() > 0 )
			result += DeleteNames[i].Len() + 1;
	}
	return result;
}

This is said byte block i used for faster name comparison, parsed from the string arrays and compacted later, I recommend defining these classes instead of passing arrays as arguments in native code, much easier to handle.

Code: Select all

class FCharObf //This is a single name entry
{
	public:
	BYTE num;
	BYTE var[68]; //4 extra bytes for flags

	FCharObf() {} //Constructor
	//Equivalence
	UBOOL operator==( const FCharObf& B ) const
	{
		if ( num != B.num )
			return false;
		for ( int i=0 ; i<num ; i++ )
			if ( var[i] != B.var[i] )
				return false;
		return true;
	}
};

struct FCharObf128 //This is a name entry array
{
	BYTE num;
	FCharObf var[128];
};

This is how I parse said blocks from 3 FString arrays:

Code: Select all

void APackageObfuscator::ParseStrings( FCharObf128 *aDelete, FCharObf128 *aReplace, FCharObf128 *aBogus)
{
	int k=0;
	for ( int i=0 ; i<128 ; i++ )
	{
		if ( DeleteNames[i].Len() > 0 )
		{
			aDelete->var[k].num = DeleteNames[i].Len() + 1;
			for ( int j=0 ; j<aDelete->var[k].num ; j++ )
				aDelete->var[k].var[j] = DeleteNames[i].GetCharArray()(j);
			k++;
		}
	}
	aDelete->num = k;

	k=0;
	for ( i=0 ; i<128 ; i++ )
	{
		if ( ObfuscateNames[i].Len() > 0 )
		{
			aReplace->var[k].num = ObfuscateNames[i].Len() + 1;
			for ( int j=0 ; j<aReplace->var[k].num ; j++ )
				aReplace->var[k].var[j] = ObfuscateNames[i].GetCharArray()(j);
			k++;
		}
	}
	aReplace->num = k;

	k=0;
	for ( i=0 ; i<128 ; i++ )
	{
		if ( BogusNames[i].Len() > 0 )
		{
			aBogus->var[k].num = BogusNames[i].Len() + 1;
			for ( int j=0 ; j<aBogus->var[k].num ; j++ )
				aBogus->var[k].var[j] = BogusNames[i].GetCharArray()(j);
			k++;
		}
	}
	aBogus->num = k;
}

Side Note: using appMemcpy function is extremely useful to copy entire arrays, I used in in FerBotz for speed reasons.
Well, any Memory handling function should be a lot faster than doing iterations to copy arrays.

[player]

i can see you are releasing a partial instead of those complete obfuscators hanging around years ago, which shows you do keep sustaining the community in mind.
but i still can't understand the motive behind locking up Siege Ultimate & each |uK| map with password variable. As I didn't ever witness mod stealing (claiming others' codes as yours) in ut99, what happened there is just like a monopoly to me. To support my view: in 2011 it is SiegeUltimateRC7, now 2013 it is SiegeUltimateRC21. I could not believe there is no single stable version in between appropriate for beta release. If that's what you use obfuscation for, then shame on you.

[[]KAOS[]Casey]

I absolutely do not approve of this topic. All obfuscating does is hurt peoples progression in programming. You'll never learn how something is performed, only that some magic box does it. Programming will become magical, rather than an understanding process.

I learned programming purely from trying to figure out how things worked with public code available from various people, I might have taken some code from some people before I truly understood how it works, but that's how progress is made. I can now code anything and everything myself without anyone's help at all, with the occasional fringe-subject I know nothing about {such as rendering,UV's, etc in my own little test game engine, but with Unreal I know pretty much everything I could possibly want to}.

All of my code is public source, any C++ mod I make has src available upon request unless I package it, UScript code is always available. I don't always package my C++ code because I may go through multiple versions quickly or I consider it unfinished, etc.

If Tim Sweeney had decided that Epic code was to be all obfuscated, where would you be now? I certainly wouldn't be here now. I certainly would not have finished college, or hell, even high school. I had no drive to do anything in high school but play & code for unreal. Without that, I would be nowhere.

Please don't ruin it for people who may really need it. You have no idea what you're doing. There's always a better way to secure your stuff on Unreal, particularly not going the security by obscurity route and actually understanding how your code could be edited and using proper validity checks.

Respect-- for Higor.

[UT99.org]

@ Casey

[Feralidragon]

but i still can't understand the motive behind locking up Siege Ultimate & each |uK| map with password variable. As I didn't ever witness mod stealing (claiming others' codes as yours) in ut99, what happened there is just like a monopoly to me. To support my view: in 2011 it is SiegeUltimateRC7, now 2013 it is SiegeUltimateRC21. I could not believe there is no single stable version in between appropriate for beta release. If that's what you use obfuscation for, then shame on you.

Let's not bring those into this topic, feel free to discuss about that in a new one please.
However, I must say this:

Spoiler

personally I know why exactly they did what they did, and there were very strong reasons. There's a lot more going under the hood than you might imagine through all communities, and there are tons of admins with ill will and which caused heavy trouble to uK and others out of jealously and anger.
They are not trying to establish a monopoly, not at all, they are actually trying to protect what they already have. Furthermore, if they are the ones developing, they have every right to hold it for as long as they want, they don't own anything to anybody. In fact, I believe it's the other way around right now.

I won't talk here about the reasons beneath their actions, so feel free to discuss it with $carface if he wishes to, and even when he intends to start making public releases because he actually wants to, but for that he has to insure quality and security of what's now the most popular mod around. I think he already explained in public some of those reasons, people just didn't buy it, but I did see in first hand what was happening.

@Casey: I know how you feel, and it's indeed true that obfuscation doesn't improve security, good well thought design does. Just like a database where you store passwords, even if you know how it works, it's still secure, it's all bound to what you actually run server-side that matters and what you validate and filter.
And like everyone knows I am all for open source and free to use code and other stuff.
However, anyone is entitled to hide their own code if they want, and we have to respect that. No developer is entitled to release anything or give it to us at all, it's their work after all which wouldn't exist if they didn't put effort on it, and thus they own it and decide what to do with it (in accordance with the EULA of course).
Furthermore, I kept hearing about obfuscators hurting the community, yet 14 years later here we are with a vast majority of coders either releasing their sources all the time, or just not giving a damn about protecton. There were only a few developers which actually used it, and in the end every single one of us are able to do a much better job than them, to not mention there are very public and easy to use tools to deobfuscate them all so you can learn anyway.

Obfuscators have been around for ages, if they didn't hurt the community so far, it's unlikely they will now imho, furthermore you can always deobfuscate them, and the ones you aren't able to, perhaps would be a good exercise to develop a tool which smartly did the exact opposite (but the ones unable too are basically the ones do not work well or at all anyway given that they corrupt too much), nullifying the whole problem.

[The_Cowboy]

I dont see any problem with the release of this tool. As Ferali said, it wont harm the community (it can never do http://forums.beyondunreal.com/showthread.php?t=193453). There are more threads at beyondunreal.
That said, I think it is a good opportunity for a guy like me to learn the obfuscation/deobfuscation (when time allows). Can we have this in tutorial section?

[Higor]

The 'security' reasons to obfuscate in this case, is to reduce the number of coders potentially able to make the exploit from 20 to 2, and to increase the time and effort they have to put into, nothing more.
Also, making a variable disappear is a good way to prevent reading/setting from another UScript module, while we have ACE doing the native checks (although not perfect).

Just because one security measure isn't perfect, it doesn't mean i should refrain from using it (aka bending over and dropping the pants) and hope for the best.
And seriously, some ppl seems to treat the obfuscation issue like inquisitors did back in the middle ages: speak and we burn you, everyone stay ignorant!!!!


====================
Back to execCreatePackage, now I'll show the native routine that handles the names themselves.
This iterator will go over all names in the name table and perform the individual checks and modifications on the DEST file.

Code: Select all

	//continued
	while ( CurNames < NameCount )
		{
			BYTE NameSize;
			BYTE NameChars[68];
			appMemzero(&NameChars, 68);
			Src->Serialize( &NameSize, 1);

			if ( NameSize != 0 ) //Because this package may have already been obfuscated
			{
				Src->Serialize( &NameChars, NameSize + 4); //Name and flags, last is always 0
				FCharObf fTmp;
				fTmp.num = NameSize;
				appMemcpy( &(fTmp.var), &NameChars, 68);
				
				//Convert into 0 length name
				if ( FoundName( &fTmp, &ToDelete) )
					Dest->Serialize( &EmptyName, 5);
				else if ( CharactersToDelete > 0 && FoundName( &fTmp, &ToBogus) )
				{
					int iTmp = 61 - fTmp.num;
					if ( iTmp > CharactersToDelete )
						iTmp = CharactersToDelete;
					INT nFlags;
					appMemcpy( &nFlags, &(fTmp.var[fTmp.num]), 4);
					AddGarbageTo( &fTmp, fTmp.num + iTmp, fTmp.num);
					appMemcpy( &(fTmp.var[fTmp.num]), &nFlags, 4);
					CharactersToDelete -= iTmp;
					Dest->Serialize( &fTmp, fTmp.num + 5); //1 + num + 4
				}
				else if ( FoundName( &fTmp, &ToReplace) )
				{
					AddGarbageTo( &fTmp, fTmp.num, 0);
					Dest->Serialize( &fTmp, fTmp.num + 5); //1 + num + 4
				}
				else
					Dest->Serialize( &fTmp, fTmp.num + 5);
			}
			else
			{
				Src->Serialize( &NameChars, 5); //Name and flags, last is always 0
				Dest->Serialize( &NameSize, 1);
				Dest->Serialize( &NameChars, 5);
			}

			CurNames++;
		}
//to be continued

Also, the member functions.

FoundName

Code: Select all

bool FoundName( FCharObf *fTest, FCharObf128 *aSrc)
{
	for ( int i=0 ; i < aSrc->num ; i++ )
		if ( aSrc->var[i] == *fTest)
			return true;
	return false;
}

AddGarbageTo, all we do here is bloat a bogus name with control characters

Code: Select all

void AddGarbageTo( FCharObf *fTest, BYTE NumChars, BYTE startFrom)
{
	fTest->num = NumChars;
	for ( int i=startFrom ; i<NumChars ; i++ )
	{
		if ( appFrand() > 0.5 )
			fTest->var[i] = 1 + (BYTE)(appFrand() * 20.0);
		else
			fTest->var[i] = 0x81 + (BYTE)(appFrand() * 19.0);
	}
	fTest->var[NumChars] = 0;
}

We already counted the characters we are going to delete, so we'll bloat bogus names by that amount of chars, even if we find the bogus names before the names we're going to delete, this is what breaks the obfuscator if we specify a name to delete that isn't on the name table.
This also handles already obfuscated names by going over them, but if we were to de-obfuscate without changing package size at all, we should start right here!

As you can see, Serialize is a VERY flexible function, i wonder if we could implement Serialize as a native function, this would enable us to do all kinds of binary load/save, any opinions?