XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby sektor2111 » Sun Jun 25, 2017 7:19 am

Here is my XC_Core.INT with help query available
Spoiler: show
[Public]
Object=(Name=XC_Core.XC_ServerCommandlet,Class=Class,MetaClass=Core.Commandlet)
Object=(Name=XC_Core.StripSourceCommandlet,Class=Class,MetaClass=Core.Commandlet)
Object=(Name=XC_Core.DeobfuscateNamesCommandlet,Class=Class,MetaClass=Core.Commandlet)
Object=(Name=XC_Core.LZMACompressCommandlet,Class=Class,MetaClass=Core.Commandlet)
Object=(Name=XC_Core.LZMADecompressCommandlet,Class=Class,MetaClass=Core.Commandlet)

[XC_ServerCommandlet]
HelpCmd=XC_Core.XC_ServerCommandlet
HelpOneLiner=Starts a dedicated server using new timer deal
HelpUsage=XC_Core.XC_ServerCommandlet server map.unr=?gametype=Package.Class?mutator=Package.Class[?parameter=value] [-options...]
HelpParm[0]=?gametype=Package.Class
HelpDesc[0]=Specifies game-type running when server is being started - example BotPack.DeathMatchPlus, else game is being read from Map used or INI file if map has no game-type info.
HelpParm[1]=?Mutator=Package.Class
HelpDesc[1]=Specifies mutators permanent loaded in any case (MapVotes muts have ones dependent on game-type - configurable separate).
HelpParm[2]=[-options...]
HelpDesc[2]=example -INI=A_File.ini Specifies INI file used by server as configuration file (Server.ini, UT.ini, etc.).
HelpParm[3]=?parameter=value
HelpDesc[3]=Example ?log=server.log - specifies main log file used by server - StatLog is another thing.

[LZMACompressCommandlet]
HelpCmd=LZMACompress
HelpOneLiner=Compress a Package using LZMA compression - requires XC_Core in client for downloading this format. A file with extension .LZMA will be created.
HelpUsage=compress File1 [File2 [File3 ...]]
HelpParm[0]=Files
HelpDesc[0]=The wildcard or file names to compress in LZMA format.

[LZMADecompressCommandlet]
HelpCmd=LZMADecompress
HelpOneLiner=Decompress a file compressed with UCC LZMACompress.
HelpUsage=decompress CompressedFile
HelpParm[0]=CompressedFile
HelpDesc[0]=The .LZMA file to decompress.

[StripSourceCommandlet]
HelpCmd=StripSource
HelpOneLiner=Strip source-code stored in a package U file
HelpUsage=StripSource U_file
HelpParm[0]=U_File
HelpDesc[0]=The .U package file for striping.

[DeobfuscateNamesCommandlet]
HelpCmd=DeobfuscateNames
HelpOneLiner=Deobfuscate source-code stored in a package U file - might crash UCC
HelpUsage=DeobfuscateNames U_file
HelpParm[0]=U_File
HelpDesc[0]=The .U package file for deobfuscating.

It looks like this
Help_Available.PNG

Edit:
Tiny corrections.
User avatar
sektor2111
Godlike
 
Posts: 4042
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby sektor2111 » Wed Aug 02, 2017 4:48 pm

Bump
I'm not sure if this is well known or not... UTPureRC7G - which I'm not using but I checked it today by chance, it seems to do funky things with XCGE so XCGE should have another deal in here because this is not helpful at all. This is a FALSE log, very False...
Code: Select all
ScriptLog: Attempted to use illegal skin from package FCommandoSkins for FCommando
ScriptLog: Failed to load FCommandoSkins.daco1 so load FCommandoSkins.cmdo1
ScriptLog: Attempted to use illegal skin from package FCommandoSkins for FCommando
ScriptLog: Failed to load FCommandoSkins.daco4Tanya so load FCommandoSkins.cmdo4
ScriptLog: Attempted to use illegal skin from package FCommandoSkins for FCommando
ScriptLog: Failed to load FCommandoSkins.cmdo1T_3 so load FCommandoSkins.cmdo1
ScriptLog: Attempted to use illegal skin from package FCommandoSkins for FCommando
ScriptLog: Failed to load FCommandoSkins.cmdo2T_3 so load FCommandoSkins.cmdo2
Log: Possessed PlayerPawn: bbTFemale1 DM-1on1-Remorc_R16.bbTFemale0
It's not about NewNet problems is old UTPure having the same issues (or it doesn't have issues...), is a wrong deal here... There is nothing illegal in that package.
User avatar
sektor2111
Godlike
 
Posts: 4042
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Wed Aug 02, 2017 8:26 pm

The UTPure skin thing 'could' be addressed... I'd have to see if it's possible to replace functions with obfuscated names.

======================================================================

As a learning experience I've been working on splitting the collision grid from XC_Engine into a separate DLL.

With a few interesting details:
Built with Visual Studio 2015, linked only against plain old MSVCRT.dll (this one took a while to figure out), with ultra slim dynamic linking as a result the base DLL size was barely 5kb when it only had a test function.
Using C++11 style coding and double checking the builds with a disassembler to make sure code output is good.
The collision library manually acquires symbols from Engine, Core instead of letting the compiler do it, this pretty much solves ANY compiler interoperability issues, making it possible for GCC 5.x.x to build it without breaking anything.
There's massive usage of SSE vector intrinsics, optimum data movement using either MOVAPS or MOV instructions (instead of FPU, MOVSS loads) even on float variables.
Eliminated 99.9% of MALLOC calls, custom element holders take care of memory management which causes AddActor/RemoveActor to operate super fast.
The grid can store actors in 3 different ways: global (referenced by grid), boundary( referenced by two or more grid elements), tree (inside a mini-octree in a grid element), this results in a change on how the query algorithms work, the Line query in particular is especially good on super long traces as the grid only traverses starting from the 'current' element into the desired direction(s, up to 3: X,Y,Z positive or negative).

Stuff to do in the future:
Assign a grid per level zone, if two grids overlap then join them and so on, this can make levels that are separated into different areas or that have LevelInfo actors run more optimally as each isolated area would have it's own grid.
Discard the PrimitiveActor system and use something less hacky, will have more details in the future.

The timing results were impressive (CTF-Warzone, 25v25), going from [2.9 to 4.2] ms/frame to [0.8 to 2.4] ms/frame.

Visual Studio 2015 can be VERY retarded at times, when working with this compiler with the intention of producing optimized code you 100% need a disassembler to check that things are ok.
This is some advanced shit over here.
Code: Select all
template<typename T> T cast(void* V)
{
   __asm   {   mov eax,V   }
}

static void SkipJump( void*& Addr)
{
   uint8* AddrByte = (uint8*)Addr;
   if ( AddrByte++[0] == 0xE9 ) //Relative long jump
   {
      int32 Offset = *((int32*)AddrByte);
      AddrByte += 4;
      AddrByte += Offset;
      Addr = (void*)AddrByte;
   }
}



Code: Select all
bool LoadUE()
{
#ifdef _WINDOWS
   // Usually the cast<T> macro would work...
   // But Visual Studio 2015 adds unnecessary memory assignments and breaks some of statics
   #define Get(dest,module,symbol) { void* A=GetProcAddress(module,symbol); __asm{ \
                           __asm mov eax,A \
                           __asm mov dest,eax } }
   // Win32 optimization: skip an unnecessary jump instruction and go to aligned memory directly
   #define GetF(dest,module,symbol) { void* A=GetProcAddress(module,symbol); SkipJump(A); __asm{ \
                           __asm mov eax,A \
                           __asm mov dest,eax } }

   {
      void* hCore = GetModuleHandle( TEXT("Core.dll"));
      void* hEngine = GetModuleHandle( TEXT("Engine.dll"));
      GetF( GetIndexedObject , hCore  , "?GetIndexedObject@UObject@@SAPAV1@H@Z");
      GetF( Debugf           , hCore  , "?Log@FOutputDevice@@QAEXPBG@Z"        );
      Get ( GLog             , hCore  , "?GLog@@3PAVFOutputDevice@@A"          );
      GetF( AppFailAssert    , hCore  , "?appFailAssert@@YAXPBD0H@Z"           );
      Get ( NameTable        , hCore  , "?Names@FName@@0V?$TArray@PAUFNameEntry@@@@A");
      Get ( GMalloc          , hCore  , "?GMalloc@@3PAVFMalloc@@A"             );
      GetF( IsMovingBrushFunc, hEngine, "?IsMovingBrush@AActor@@QBEHXZ"        );
   }

#elif __GNUC__
//NOT FINISHED OR TESTED, STILL DOCUMENTING THE SYMBOLS
   GetIndexedObject = cast<vp_func_i>( dlsym(RTLD_DEFAULT, "GetIndexedObject__7UObjecti"));
   Debugf = cast<v_foutputdevice_func_cp>( dlsym(RTLD_DEFAULT, "Log__13FOutputDevicePCc"));
   GLog = cast<foutputdevicepp_var>( dlsym(RTLD_DEFAULT, "GLog"));
   AppFailAssert = cast<v_func_acp_acp_i>( dlsym(RTLD_DEFAULT, "appFailAssert__FPCcT0i"));
   NameTable = cast<fnametableppp_var>( dlsym(RTLD_DEFAULT, "_5FName.Names"));
   GMalloc = cast<fmallocpp_var>( dlsym(RTLD_DEFAULT, "GMalloc"));
   IsMovingBrushFunc = cast<i_aactor_v>( dlsym(RTLD_DEFAULT, "IsMovingBrush__C6AActor"));
#endif

   Loaded++;
   return GetIndexedObject && Debugf && GLog && AppFailAssert && NameTable && GMalloc && IsMovingBrushFunc;
}



EDIT: CheatEngine is a world class debugger if you know how to halt the game before the game crashes in order to allow you to attach the debugger, usually with an infinite loop lol.
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby sektor2111 » Wed Aug 02, 2017 8:42 pm

It sounds like good news, so you can even do disassembling and checking initial integrity - of course if coding method is OK.
By chance, should I expect some new deals in DevPath ? I'm just thinking at that cycles crap and 1000 paths... I would like to see a real CPU based job. My machine probably can find 2000 Paths, HOF server being stronger might go over 8000 paths and so on - 1000 paths are hard-coded for years '99 it's probably time for a dynamic value dependent on CPU power. A new time vs cycles deal would be priceless generally using more time when the CPU is idling useless with NOPs... eh... not to mention that natives probably can use multiple cores if I'm not mistaking...
User avatar
sektor2111
Godlike
 
Posts: 4042
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Wed Aug 02, 2017 9:15 pm

Hotpatching code in Linux isn't goint to be that easy, unfortunately.
I'd rather go with vftable hooks everywhere.
And with paths, the route mapper can be so much better (especially the optimized/unreleased one lol).

This is the full anomaly list after 40 minutes of 25v25 gameplay, no crashes, kinda surprised at not seeing so many of them.
Still some Botz appear to get sent into the infinite after some hits lol, gotta figure out what's up with the traces.
Also most of the CollisionTag related anomalies are triggered by the game's physics trying to add/remove actors more than once, this could be responsible for some of the crashes both in the old hash and grid.

Code: Select all
XC_Engine: Setting up collision octree [Level Hook] for CTF-WarZone
Log: [CG] Element holders succesfully spawned.
Log: [CG] Grid allocated [FULL].
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] Allocating extra element holder for ActorInfo
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] Invalid actor location: Botz45
Log: Botz45 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget2
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] Invalid actor location: Botz26
Log: Botz26 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget6
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget45
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget3
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget2
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget4
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget5
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget7
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget3
Log: [CG] Invalid actor location: Botz43
Log: Botz43 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget14
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget1
Log: [CG] Invalid actor location: Botz52
Log: Botz52 fell out of the world!
Log: [CG] Invalid actor location: Botz48
Log: Botz48 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget0
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget0
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget5
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget6
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget8
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget9
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget2
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget3
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget1
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget2
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] Invalid actor location: Botz10
Log: Botz10 fell out of the world!
Log: [CG] Invalid actor location: Botz6
Log: Botz6 fell out of the world!
Log: [CG] Invalid actor location: Botz52
Log: Botz52 fell out of the world!
Log: [CG] Invalid actor location: Botz8
Log: Botz8 fell out of the world!
Log: [CG] Invalid actor location: Botz1
Log: Botz1 fell out of the world!
Log: [CG] Invalid actor location: Botz52
Log: Botz52 fell out of the world!
Log: [CG] Invalid actor location: Botz34
Log: Botz34 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget15
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget17
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget6
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget3
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget9
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget10
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget13
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget14
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Invalid actor location: Botz9
Log: Botz9 fell out of the world!
Log: [CG] Invalid actor location: Botz20
Log: Botz20 fell out of the world!
Log: [CG] Invalid actor location: Botz27
Log: Botz27 fell out of the world!
Log: [CG] Invalid actor location: Botz27
Log: Botz27 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget7
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget2
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Error in RemoveActorInfo: Link at CurDepth=1 is non-existant, Actor is UTChunk5
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Invalid actor location: Botz53
Log: Botz53 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget65
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget5
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget8
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget10
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget6
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget46
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget7
Log: [CG] Invalid actor location: Botz14
Log: Botz14 fell out of the world!
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for TranslocatorTarget1
Log: [CG] Invalid actor location: Botz41
Log: Botz41 fell out of the world!
Log: [CG] Invalid actor location: Botz0
Log: Botz0 fell out of the world!
Log: [CG] Invalid actor location: Botz54
Log: Botz54 fell out of the world!
Log: [CG] Invalid actor location: Botz10
Log: Botz10 fell out of the world!
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget22
Log: [CG] Error in RemoveActorInfo: Link at CurDepth=3 is non-existant, Actor is UTChunk79
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Anomaly in InsertActor: CollisionTag not zero for BotzTTarget68
Log: [CG] Invalid actor location: Botz51
Log: Botz51 fell out of the world!
Log: [CG] Allocating extra element holder for MiniTree
Log: [CG] ActorInfo::IsValid -> Using invalid memory
Log: [CG] Invalid actor location: Botz44
Log: Botz44 fell out of the world!
Log: [CG] Invalid actor location: Botz53
Log: Botz53 fell out of the world!
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby sektor2111 » Thu Aug 03, 2017 6:22 am

Higor wrote:Log: [CG] Error in RemoveActorInfo: Link at CurDepth=1 is non-existant, Actor is UTChunk5
Well, what I got weeks ago was pretty much closer to this - conforming some files was the task - borked client spoke about expecting some object and... getting another class like array was corrupted somehow... :???:
As another fact at this moment I'm not using bUseLevelHook, I could see funky colliding ghosts so I had to disable it, effects at a moment are strange... CollisionGrid has probably hidden glitches.
User avatar
sektor2111
Godlike
 
Posts: 4042
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Chris » Thu Aug 03, 2017 6:45 pm

Higor wrote:The UTPure skin thing 'could' be addressed... I'd have to see if it's possible to replace functions with obfuscated names.

======================================================================

As a learning experience I've been working on splitting the collision grid from XC_Engine into a separate DLL.

With a few interesting details:
Built with Visual Studio 2015, linked only against plain old MSVCRT.dll (this one took a while to figure out), with ultra slim dynamic linking as a result the base DLL size was barely 5kb when it only had a test function.
Using C++11 style coding and double checking the builds with a disassembler to make sure code output is good.
The collision library manually acquires symbols from Engine, Core instead of letting the compiler do it, this pretty much solves ANY compiler interoperability issues, making it possible for GCC 5.x.x to build it without breaking anything.
There's massive usage of SSE vector intrinsics, optimum data movement using either MOVAPS or MOV instructions (instead of FPU, MOVSS loads) even on float variables.
Eliminated 99.9% of MALLOC calls, custom element holders take care of memory management which causes AddActor/RemoveActor to operate super fast.
The grid can store actors in 3 different ways: global (referenced by grid), boundary( referenced by two or more grid elements), tree (inside a mini-octree in a grid element), this results in a change on how the query algorithms work, the Line query in particular is especially good on super long traces as the grid only traverses starting from the 'current' element into the desired direction(s, up to 3: X,Y,Z positive or negative).

Stuff to do in the future:
Assign a grid per level zone, if two grids overlap then join them and so on, this can make levels that are separated into different areas or that have LevelInfo actors run more optimally as each isolated area would have it's own grid.
Discard the PrimitiveActor system and use something less hacky, will have more details in the future.

The timing results were impressive (CTF-Warzone, 25v25), going from [2.9 to 4.2] ms/frame to [0.8 to 2.4] ms/frame.

Visual Studio 2015 can be VERY retarded at times, when working with this compiler with the intention of producing optimized code you 100% need a disassembler to check that things are ok.
This is some advanced shit over here.
Code: Select all
template<typename T> T cast(void* V)
{
   __asm   {   mov eax,V   }
}

static void SkipJump( void*& Addr)
{
   uint8* AddrByte = (uint8*)Addr;
   if ( AddrByte++[0] == 0xE9 ) //Relative long jump
   {
      int32 Offset = *((int32*)AddrByte);
      AddrByte += 4;
      AddrByte += Offset;
      Addr = (void*)AddrByte;
   }
}



Code: Select all
bool LoadUE()
{
#ifdef _WINDOWS
   // Usually the cast<T> macro would work...
   // But Visual Studio 2015 adds unnecessary memory assignments and breaks some of statics
   #define Get(dest,module,symbol) { void* A=GetProcAddress(module,symbol); __asm{ \
                           __asm mov eax,A \
                           __asm mov dest,eax } }
   // Win32 optimization: skip an unnecessary jump instruction and go to aligned memory directly
   #define GetF(dest,module,symbol) { void* A=GetProcAddress(module,symbol); SkipJump(A); __asm{ \
                           __asm mov eax,A \
                           __asm mov dest,eax } }

   {
      void* hCore = GetModuleHandle( TEXT("Core.dll"));
      void* hEngine = GetModuleHandle( TEXT("Engine.dll"));
      GetF( GetIndexedObject , hCore  , "?GetIndexedObject@UObject@@SAPAV1@H@Z");
      GetF( Debugf           , hCore  , "?Log@FOutputDevice@@QAEXPBG@Z"        );
      Get ( GLog             , hCore  , "?GLog@@3PAVFOutputDevice@@A"          );
      GetF( AppFailAssert    , hCore  , "?appFailAssert@@YAXPBD0H@Z"           );
      Get ( NameTable        , hCore  , "?Names@FName@@0V?$TArray@PAUFNameEntry@@@@A");
      Get ( GMalloc          , hCore  , "?GMalloc@@3PAVFMalloc@@A"             );
      GetF( IsMovingBrushFunc, hEngine, "?IsMovingBrush@AActor@@QBEHXZ"        );
   }

#elif __GNUC__
//NOT FINISHED OR TESTED, STILL DOCUMENTING THE SYMBOLS
   GetIndexedObject = cast<vp_func_i>( dlsym(RTLD_DEFAULT, "GetIndexedObject__7UObjecti"));
   Debugf = cast<v_foutputdevice_func_cp>( dlsym(RTLD_DEFAULT, "Log__13FOutputDevicePCc"));
   GLog = cast<foutputdevicepp_var>( dlsym(RTLD_DEFAULT, "GLog"));
   AppFailAssert = cast<v_func_acp_acp_i>( dlsym(RTLD_DEFAULT, "appFailAssert__FPCcT0i"));
   NameTable = cast<fnametableppp_var>( dlsym(RTLD_DEFAULT, "_5FName.Names"));
   GMalloc = cast<fmallocpp_var>( dlsym(RTLD_DEFAULT, "GMalloc"));
   IsMovingBrushFunc = cast<i_aactor_v>( dlsym(RTLD_DEFAULT, "IsMovingBrush__C6AActor"));
#endif

   Loaded++;
   return GetIndexedObject && Debugf && GLog && AppFailAssert && NameTable && GMalloc && IsMovingBrushFunc;
}



EDIT: CheatEngine is a world class debugger if you know how to halt the game before the game crashes in order to allow you to attach the debugger, usually with an infinite loop lol.


Yet M$ just love to believe that compilers always generate better assembly than humans, which is why targeting x64 in MSVC is a pain and sometimes impossible when dealing with low-level tasks.
Small thought on your cast function, isn't the whole point of it to be a dynamic cast? Unless the point of this one (in lower case) is to remove the overhead of type safety, but then shouldn't a C-style cast be sufficient rather than jumping to a new function?
Chris
Experienced
 
Posts: 116
Joined: Mon Nov 24, 2014 9:27 am

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Thu Aug 03, 2017 11:11 pm

Try typecasting from (void*) to a typedef'd function pointer and see what happens.
Even the cast<> template started failing after VS decided to add out of nowhere a MOVUPS into a chunk of memory that overlapped with one of my global function pointers.
So these 'Get' and 'GetF' macros are the cleanest way I could come up with where the compiler decided to not do stuff I didn't tell it to do.

Interesting stuff, both Core.dll and Engine.dll have some of their exported functions in a global table, which consists basically on:
- align code to 4 bytes
- E9 far jump to where the function actually is.
- rinse and repeat with a lot other functions.
So the GetF macro not only gets the function address (on this table), it also skips that unnecessary jump by setting the address to the jump destination (which is 32-aligned) making the code a cycle or two faster.
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Chris » Fri Aug 04, 2017 12:39 am

Higor wrote:Try typecasting from (void*) to a typedef'd function pointer and see what happens.
Even the cast<> template started failing after VS decided to add out of nowhere a MOVUPS into a chunk of memory that overlapped with one of my global function pointers.
So these 'Get' and 'GetF' macros are the cleanest way I could come up with where the compiler decided to not do stuff I didn't tell it to do.

Interesting stuff, both Core.dll and Engine.dll have some of their exported functions in a global table, which consists basically on:
- align code to 4 bytes
- E9 far jump to where the function actually is.
- rinse and repeat with a lot other functions.
So the GetF macro not only gets the function address (on this table), it also skips that unnecessary jump by setting the address to the jump destination (which is 32-aligned) making the code a cycle or two faster.


I set up a small test as such:

Code: Select all
int TestFunc()
{
   return 100;
}
//MAIN:
      void* fptr;
      fptr = &TestFunc;

      typedef int (*tfn)(void);

      tfn funcs = (tfn)fptr;
      std::cout << (*funcs)() << '\n';


Worked just fine @ VS2010 targeting x64 (Debug and Release mode with and without OPT:REF & OPT:ICF).

I haven't bothered using anything newer and from the looks of it maybe I shouldn't.

Hmm isn't E9 a relative near jump opcode? EA would be a far jump to an absolute address. Is this something the old compiler they used back then messed up?
Chris
Experienced
 
Posts: 116
Joined: Mon Nov 24, 2014 9:27 am

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Fri Aug 04, 2017 11:09 pm

Now get the function pointer using GetProcAddress, on x86.
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Chris » Sat Aug 05, 2017 6:17 am

I haven't tried that yet, but I am curious as to why you'd dynamically link to the libraries rather than statically? It would save you quite a bit of processing time and it would get rid of the extra indirection.
Your GetF makes sense in what it does, but wouldn't it be possible to get to the desired address right away by using the __imp__ prefix rather than jumping to the indirect dereference function?
Say something like "__imp_?GetIndexedObject@UObject@@SAPAV1@H@Z". I haven't tried it yet, it's more of a guess based on how a dll is designed. Hopefully I don't sound too ignorant.
Chris
Experienced
 
Posts: 116
Joined: Mon Nov 24, 2014 9:27 am

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Sat Aug 05, 2017 5:49 pm

The compiler's dynamic linking is that, exactly the same of what I'm doing, that is because the compiler adds an entry point where it does this 'implicitly'.
I'm doing it manually because I plan to make this DLL entirely compiler independent, with a few #defines here and there it could be possible to build this using GCC 5, ICC, VS 2017, MinGW, etc. All without losing compatibility with UT99 binaries.

Btw this is an example of how Engine.dll exports it's functions.
They're all in a table with jumps, I don't know why the dll was built this way, but the GetF macro simply skips this step at load time and directly calls the function where it actually is.
Code: Select all
103039A4                                                       ?SetWorldRaytraceKey@AMover@@UAEXXZ:
103039A4  E9778C0A00                                                  jmp   L103AC620
103039A9                                                       ??0AHUD@@IAE@XZ:
103039A9  E9D2990300                                                  jmp   L1033D380
103039AE                                                       ?eventTakeDamage@AActor@@QAEXHPAVAPawn@@VFVector@@1VFName@@@Z:
103039AE  E92D1E0100                                                  jmp   L103157E0
103039B3                                                       ?SetLastUpdateTime@UBitmap@@QAEXVFTime@@@Z:
103039B3  E928F60000                                                  jmp   L10312FE0
Last edited by Higor on Sat Aug 05, 2017 6:02 pm, edited 1 time in total.
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby sektor2111 » Sat Aug 05, 2017 5:59 pm

I'm ready for x86 testing if you have some cute DLL for probing...
User avatar
sektor2111
Godlike
 
Posts: 4042
Joined: Sun May 09, 2010 6:15 pm
Location: On the roof.

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Chris » Sat Aug 05, 2017 7:47 pm

Higor wrote:The compiler's dynamic linking is that, exactly the same of what I'm doing, that is because the compiler adds an entry point where it does this 'implicitly'.
I'm doing it manually because I plan to make this DLL entirely compiler independent, with a few #defines here and there it could be possible to build this using GCC 5, ICC, VS 2017, MinGW, etc. All without losing compatibility with UT99 binaries.

Btw this is an example of how Engine.dll exports it's functions.
They're all in a table with jumps, I don't know why the dll was built this way, but the GetF macro simply skips this step at load time and directly calls the function where it actually is.
Code: Select all
103039A4                                                       ?SetWorldRaytraceKey@AMover@@UAEXXZ:
103039A4  E9778C0A00                                                  jmp   L103AC620
103039A9                                                       ??0AHUD@@IAE@XZ:
103039A9  E9D2990300                                                  jmp   L1033D380
103039AE                                                       ?eventTakeDamage@AActor@@QAEXHPAVAPawn@@VFVector@@1VFName@@@Z:
103039AE  E92D1E0100                                                  jmp   L103157E0
103039B3                                                       ?SetLastUpdateTime@UBitmap@@QAEXVFTime@@@Z:
103039B3  E928F60000                                                  jmp   L10312FE0

It almost looks like the behavior of the import table where the dynamic linker of Windows would fill in the addresses as it loads the image.
Is this an anomaly from what is described here? http://blog.omega-prime.co.uk/?p=115
Is there any performance differences between a relative jump and an absolute one? I guess it shouldn't be.
Chris
Experienced
 
Posts: 116
Joined: Mon Nov 24, 2014 9:27 am

Re: XC_Engine [20] - XC_Core [7b] - XC_IpDrv

Postby Higor » Sat Aug 05, 2017 8:21 pm

I have absolutely no idea, bear in mind that old C++ compilers didn't follow standards that well.
Take a look at GCC, the 2.95 version was putting the vftable pointers at the end of structs, it just worked with UObject because UObject is a subclass of FUnknown (no variables, vftable at offset 0)... maybe this is exactly one of the reasons they did it that way.

What the relative jump does is simplify DLL loading, when you put the code into memory you don't need to do weird address translations/loading or anything when you're working inside the same dll, the relative jumps works great there, absolute addresses are more of a function call thing.
Newer compilers have the symbols table direct exactly to where the functions are, not to a weird middle-man table with jumps which is why I wonder it was done that way.


Back to XC_Engine, there's still some unstable behaviour to take care of, as well as splitting the map in various local grids.
Higor
Godlike
 
Posts: 1759
Joined: Sun Mar 04, 2012 6:47 pm

PreviousNext

Return to Discussions

Who is online

Users browsing this forum: No registered users and 1 guest

cron