XC_Engine megathread
- sektor2111
- Godlike
- Posts: 6418
- Joined: Sun May 09, 2010 6:15 pm
- Location: On the roof.
Re: XC_Engine megathread
ALL are loaded fast if you have been fired game previously. At first launch after system's boot up, if you have 15+ game-types, starting a "Practice Session" takes a while (I was about to sleep) until menu shows up on a slower machine with a slower hard drive and nothing can speed up this because that disk has limited reading speed regarding to what you do. This is why I'm using a sort of "precache" task in Win servers in order to gain files placement (files list) for "DynamicLoad" specific mods... else all it's a lagging fest until OS is caching data.
Re: XC_Engine megathread
mmm...understoodsektor2111 wrote:ALL are loaded fast if you have been fired game previously. At first launch after system's boot up, if you have 15+ game-types, starting a "Practice Session" takes a while (I was about to sleep) until menu shows up on a slower machine with a slower hard drive and nothing can speed up this because that disk has limited reading speed regarding to what you do. This is why I'm using a sort of "precache" task in Win servers in order to gain files placement (files list) for "DynamicLoad" specific mods... else all it's a lagging fest until OS is caching data.
Re: XC_Engine megathread
The main problem with the map list combobox is the sorting algorithm.
UnrealScript is simply bad at sorting due to the speed of the virtual machine and the iteration limits that crash the game (safety assert actually, not a crash)
I remind you that GetMapName function is used to retrieve ONE map at a time, so you have to call it 'N' times for 'N' maps.
The new change involves XC_Engine_Menu disabling any UnrealScript sorting whatsoever.
So after disabling the sorting, the map list is constructed almost instantly...
Why was it done in the first place?
Maybe because there is no sorting in the native 'GetMapName' function as it gets the maps in the order the operating system supplies them.
Why not sorting in GetMapName? (c++)
Remember, GetMapName needs to be called 'N' times, and UT's default GetMapName function retrieves the map names using operating system API (win32 and linux).
The map list may not be sorted due to multi-folder (many map folders), also in Linux the map names aren't even sorted by name at all.
So how did I fix this problem?
Single-frame cache that gets cleared every time a frame is rendered (or the world is updated on a server).
So the first time I call GetMapName (XC version), the map list is constructed and sorted accordingly.
The results are then stored in a map list cache, as well as a marker of the position of the last map that has been returned (initially 0)
Every time I call GetMapName to get the other maps (N maps = N calls), the position marker changes to 'next' or 'prev' it simply takes the map from the cache.
The XC GetMapName's cache won't be rebuilt if GetMapName is called as follows:
The above examples should cover pretty much 99% of cases.
Result is: no UnrealScript sorting, and a one-time sorting everytime we want to generate a map list.
EDIT, play in 1080p:
UnrealScript is simply bad at sorting due to the speed of the virtual machine and the iteration limits that crash the game (safety assert actually, not a crash)
I remind you that GetMapName function is used to retrieve ONE map at a time, so you have to call it 'N' times for 'N' maps.
The new change involves XC_Engine_Menu disabling any UnrealScript sorting whatsoever.
So after disabling the sorting, the map list is constructed almost instantly...
Why was it done in the first place?
Maybe because there is no sorting in the native 'GetMapName' function as it gets the maps in the order the operating system supplies them.
Why not sorting in GetMapName? (c++)
Remember, GetMapName needs to be called 'N' times, and UT's default GetMapName function retrieves the map names using operating system API (win32 and linux).
The map list may not be sorted due to multi-folder (many map folders), also in Linux the map names aren't even sorted by name at all.
So how did I fix this problem?
Single-frame cache that gets cleared every time a frame is rendered (or the world is updated on a server).
So the first time I call GetMapName (XC version), the map list is constructed and sorted accordingly.
The results are then stored in a map list cache, as well as a marker of the position of the last map that has been returned (initially 0)
Every time I call GetMapName to get the other maps (N maps = N calls), the position marker changes to 'next' or 'prev' it simply takes the map from the cache.
The XC GetMapName's cache won't be rebuilt if GetMapName is called as follows:
Code: Select all
GetMapName( "CTF", FirstMap, Position); //Position increments with every call
GetMapName( "CTF", CurrentMap, 1); //CurrentMap changes with every GetMapName call, but '1' indicates direction to follow (next)
GetMapName( "CTF", CurrentMap, -1); //CurrentMap changes with every GetMapName call, but '-1' indicates direction to follow (prev)
Result is: no UnrealScript sorting, and a one-time sorting everytime we want to generate a map list.
EDIT, play in 1080p:
Re: XC_Engine megathread
Script compiler.
As seen here: https://github.com/CacoFFF/XC-UT99/comm ... 22d6f1e212
On the upcoming XC_Core version, the script compiler will receive a hack (as long as XC_Core is loaded) where most XC_Engine/XC_Core
functions will become global methods that no longer need to be defined in any class.
What does it mean?
If you want to make a class use XC_Engine functions, you no longer have to copy the function definitions over to the new class.
With this comes an interesting side effect that is that you can also call these functions on already existing classes.
Example where you're calling these functions without defining them, both in 'self' and 'P'.
As seen here: https://github.com/CacoFFF/XC-UT99/comm ... 22d6f1e212
On the upcoming XC_Core version, the script compiler will receive a hack (as long as XC_Core is loaded) where most XC_Engine/XC_Core
functions will become global methods that no longer need to be defined in any class.
What does it mean?
If you want to make a class use XC_Engine functions, you no longer have to copy the function definitions over to the new class.
With this comes an interesting side effect that is that you can also call these functions on already existing classes.
Example where you're calling these functions without defining them, both in 'self' and 'P'.
Code: Select all
class Test expands Object;
static function int CountDynamic( PlayerPawn P, out float Time)
{
local float F[2];
local Actor A;
local int i;
Clock(F);
ForEach P.DynamicActors( class'Actor', A)
i++;
Time = UnClock(F);
}
- sektor2111
- Godlike
- Posts: 6418
- Joined: Sun May 09, 2010 6:15 pm
- Location: On the roof.
Re: XC_Engine megathread
I must ask: What minimal specifications must have the CPU running this XC_Core ? I'm asking because XCv23 doesn't work in all my machines and I do not have plans to throw money for upgrading 4 machines and... the rest of my needs...
Re: XC_Engine megathread
Post full specs of the systems having problems with XC. (mainly operating system version/distro and CPU name, arch)
I suspect it's the c++11 requirements in CacusLib, which I'll deal with accordingly.
I suspect it's the c++11 requirements in CacusLib, which I'll deal with accordingly.
- sektor2111
- Godlike
- Posts: 6418
- Joined: Sun May 09, 2010 6:15 pm
- Location: On the roof.
Re: XC_Engine megathread
All PIV machines are running until a random moment when game is crashing (daughter is playing locally multiple times the same map) XCv23 - What it's often seen (or not seen) when a dependent file/object is missing, game is not returning to entry map, but crashing... launcher used is default because the newer it's not what I want, it was more unstable and wants to send an "error report" bla bla.
AMD proudly presented here... viewtopic.php?f=64&t=11630&start=0#p94126 being
it's not even blinking in XCv23, all backward compatibility it's gone - this was one of my best machines for playing UT'99 in back days without X cores problems and all that circus. Even if I have here more resources and I could use last XC_Core, I cannot do anything, I see things unstable and that's why I did not do any configurable UScript shared publicly but private versions for testing.
AMD proudly presented here... viewtopic.php?f=64&t=11630&start=0#p94126 being
it's not even blinking in XCv23, all backward compatibility it's gone - this was one of my best machines for playing UT'99 in back days without X cores problems and all that circus. Even if I have here more resources and I could use last XC_Core, I cannot do anything, I see things unstable and that's why I did not do any configurable UScript shared publicly but private versions for testing.
Re: XC_Engine megathread
Well well, decided to take a look at the generated code and found something interesting:
This
Was producing this, moving numbers between xmm and fpu registers, with a SSE2 DOUBLE->INT conversion due to appSqrt returning a DOUBLE instead of FLOAT.
So a small redefinition of appSqrt that takes a FLOAT and returns a FLOAT, couple more tweaks and tada:
Now I'm gonna have to hunt for more SSE2 instructions all over the code generation, and find out a way to get rid of those unnecessary XORPS instructions
This
Code: Select all
Path->visitedWeight = appRound( appSqrt((*Ptr)->Dist2DSq));
Code: Select all
fld dword ptr [eax+0Ch]
sub esp, 8
fstp qword ptr [esp+08h]
call ds:__imp_?appSqrt@@YANN@Z ; appSqrt(double)
add esp, 8
fstp [ebp-44h]
cvtsd2si eax, [ebp-44h] //SSE2 instruction
mov [ebx+314h], eax
Code: Select all
movss xmm0, dword ptr [eax+0Ch]
movss [ebp-38h], xmm0
xorps xmm1, xmm1
movss xmm1, xmm0
sqrtss xmm1, xmm1
movss [ebp-30h], xmm1
xorps xmm0, xmm0
movss xmm0, xmm1
cvtss2si eax, xmm0 //SSE instruction
mov [edx+314h], eax
- sektor2111
- Godlike
- Posts: 6418
- Joined: Sun May 09, 2010 6:15 pm
- Location: On the roof.
Re: XC_Engine megathread
And after that you might want to look at "Mover history". Is everything in these movers initialized properly ? I did a test-map with a door and buttons that need to be shot for opening door. If human player will shot them first time, Bot will suddenly know that handling, else they don't look like they know what to do during firsts seconds. Why I have to speed up using door first ? I don't know. Does Navigation need a bit of delay ? At a moment they were shooting door and I was only witnessing their roaming doing nothing because these were working properly - maybe I ruined something and I don't see what... I go to switch machines and I'll drop here that lab-junk, maybe some people will understand that shotable stuff is doable as long as Bot code can do some stunts even if UT don't have such things in original stock.
This is the thing in cause. Using 2 Freelanced Bots.
[attachment=0]MH-ShotOpen.7z[/attachment]
This is the thing in cause. Using 2 Freelanced Bots.
- Attachments
-
- MH-ShotOpen.7z
- (24.81 KiB) Downloaded 29 times
Re: XC_Engine megathread
Trying to execute unsupported instructions usually results in a hardware exception being raised and passed onto the OS. The OS would then say something like "illegal instruction".sektor2111 wrote:All PIV machines are running until a random moment when game is crashing (daughter is playing locally multiple times the same map) XCv23 - What it's often seen (or not seen) when a dependent file/object is missing, game is not returning to entry map, but crashing... launcher used is default because the newer it's not what I want, it was more unstable and wants to send an "error report" bla bla.
AMD proudly presented here... viewtopic.php?f=64&t=11630&start=0#p94126 being
it's not even blinking in XCv23, all backward compatibility it's gone - this was one of my best machines for playing UT'99 in back days without X cores problems and all that circus. Even if I have here more resources and I could use last XC_Core, I cannot do anything, I see things unstable and that's why I did not do any configurable UScript shared publicly but private versions for testing.
Is there any textual info in the error report mentioning unsupported instructions?
Visual C++ "should" never generate SSE2 under x86 unless specifically instructed to do so with the "Enable enhanced instruction set" found in the project config.Higor wrote:Well well, decided to take a look at the generated code and found something interesting:
ThisWas producing this, moving numbers between xmm and fpu registers, with a SSE2 DOUBLE->INT conversion due to appSqrt returning a DOUBLE instead of FLOAT.Code: Select all
Path->visitedWeight = appRound( appSqrt((*Ptr)->Dist2DSq));
So a small redefinition of appSqrt that takes a FLOAT and returns a FLOAT, couple more tweaks and tada:Code: Select all
fld dword ptr [eax+0Ch] sub esp, 8 fstp qword ptr [esp+08h] call ds:__imp_?appSqrt@@YANN@Z ; appSqrt(double) add esp, 8 fstp [ebp-44h] cvtsd2si eax, [ebp-44h] //SSE2 instruction mov [ebx+314h], eax
Now I'm gonna have to hunt for more SSE2 instructions all over the code generation, and find out a way to get rid of those unnecessary XORPS instructionsCode: Select all
movss xmm0, dword ptr [eax+0Ch] movss [ebp-38h], xmm0 xorps xmm1, xmm1 movss xmm1, xmm0 sqrtss xmm1, xmm1 movss [ebp-30h], xmm1 xorps xmm0, xmm0 movss xmm0, xmm1 cvtss2si eax, xmm0 //SSE instruction mov [edx+314h], eax
I'm not sure if the default "Not set" makes the compiler think that it can use whatever it wants. What if you specifically select SSE or IA32(x87 only) rather than "Not set"?
Re: XC_Engine megathread
I added a variation of appRound() that directly rounds double floats using a CVTSD2SI instead of converting to float first, left it lying there doing 'nothing' because why not.Chris wrote:Visual C++ "should" never generate SSE2 under x86 unless specifically instructed to do so with the "Enable enhanced instruction set" found in the project config.
But thanks to taking a second look I saw that the compiler is picking it up after any of the 'app' functions that work on doubles, a very stupid mistake from my part.
In XC_Core and derivates VC++ is not using any of the special built-in type conversions, it's mostly handled manually through platform specific code.
Example, compare FTime implementations (mine vs original):
https://github.com/CacoFFF/XC-UT99/blob ... ore/Core.h
https://github.com/stephank/surreal/blo ... Inc/Core.h
One of the reasons to do this, is to discard the usage of the VC140 runtime libraries when switching to the new compiler, binary stays small and nobody needs to install the new runtimes.
https://github.com/CacoFFF/XC-UT99/blob ... /API_MSC.h
But I still have to find a workaround to the FTOL2 conversion, right now it only uses SSE2 instructions.
PD: This method keeps the exception handlers consistent with the original libraries, new binaries can also add their function history during a crash.
Re: XC_Engine megathread
I've also encountered the mess caused by the poor versioning system with the CRT binaries.Higor wrote:I added a variation of appRound() that directly rounds double floats using a CVTSD2SI instead of converting to float first, left it lying there doing 'nothing' because why not.Chris wrote:Visual C++ "should" never generate SSE2 under x86 unless specifically instructed to do so with the "Enable enhanced instruction set" found in the project config.
But thanks to taking a second look I saw that the compiler is picking it up after any of the 'app' functions that work on doubles, a very stupid mistake from my part.
In XC_Core and derivates VC++ is not using any of the special built-in type conversions, it's mostly handled manually through platform specific code.
Example, compare FTime implementations (mine vs original):
https://github.com/CacoFFF/XC-UT99/blob ... ore/Core.h
https://github.com/stephank/surreal/blo ... Inc/Core.h
One of the reasons to do this, is to discard the usage of the VC140 runtime libraries when switching to the new compiler, binary stays small and nobody needs to install the new runtimes.
https://github.com/CacoFFF/XC-UT99/blob ... /API_MSC.h
But I still have to find a workaround to the FTOL2 conversion, right now it only uses SSE2 instructions.
PD: This method keeps the exception handlers consistent with the original libraries, new binaries can also add their function history during a crash.
The most common one is probably libraries being linked with different versions having different malloc implementations.
The different exception handlers is another one, as you mentioned.
So you pretty much have to resort to the FPU for the FTOL2 conversion?
You could either change the FPU control word bit 10, 11 to 0b11 and then perform the
Code: Select all
fld [esp]
fistp [esp + 8] ;Or wherever you wish to store it
mov eax, dword ptr[esp + 8]
- sektor2111
- Godlike
- Posts: 6418
- Joined: Sun May 09, 2010 6:15 pm
- Location: On the roof.
Re: XC_Engine megathread
I've seen video (I went ahead to subscribe to that channel on App which I have on the phone - more easy to handle any future videos).
I'm curious if those compilers will deliver some result which I'm expecting.
For the record M.G.S. ( aka MonsterGamingServer ) has stopped provider service feature for restarting server daily at 2:00 AM. I've added an internal restart module using console for this task - purpose is to have at least old LOGS, previously ruined by their "KILL-TASK" things. The problem is that server from now on is... crashing when console is saying EXIT - QUIT because certain things are "changed" in XC23. The Goal was to gain a restart not a crash...
But... I'm curious about newer DevPath as well...
I'm curious if those compilers will deliver some result which I'm expecting.
For the record M.G.S. ( aka MonsterGamingServer ) has stopped provider service feature for restarting server daily at 2:00 AM. I've added an internal restart module using console for this task - purpose is to have at least old LOGS, previously ruined by their "KILL-TASK" things. The problem is that server from now on is... crashing when console is saying EXIT - QUIT because certain things are "changed" in XC23. The Goal was to gain a restart not a crash...
But... I'm curious about newer DevPath as well...
Re: XC_Engine megathread
Having some fun here after someone gave me a nice idea.
Had to implement my own version of the following U227 opcode
In order to use the preload and listing required to achieve the following:
On a different note, the class XC_EditorLoader has become an important class for the hacked script compiler since it contains the XC_Core function templates to be used by the compiler (so you don't need to write the function declarations).
Additionally, the dynamic array handlers are generic and don't require subtype definitions anymore.
This works on any part of your code, as long as XC_Core and XC_Engine are on the EditPackages:
Had to implement my own version of the following U227 opcode
Code: Select all
// Load an enitre package to memory.
// @ PackageName - The name of package to load.
// @ ListType - Type of objects it should list.
// @ PckContents - The list of objects it found from it.
native(257) static final function bool LoadPackageContents( string PackageName, Class<Object> ListType, out array<Object> PckContents );
Additionally, the dynamic array handlers are generic and don't require subtype definitions anymore.
Code: Select all
native(640) static final function int Array_Length( out array<Object> Ar, optional int SetSize);
native(641) static final function bool Array_Insert( out array<Object> Ar, int Offset, optional int Count );
native(642) static final function bool Array_Remove( out array<Object> Ar, int Offset, optional int Count );
Code: Select all
local array<int> Data;
local int i, Total;
for ( i=Array_Length(Data)-1 ; i>=0 ; i-- )
Total += Data[i];