nodeJS UTStats

[UT Sniper (SJA94)]

The last few days I've been working on a new utstats system that will use nodeJS instead of php, one thing I've been struggling to make my mind up on is whether I'm going to do one complete package, or do a server module, combined with a website module.

Server module:
This will handle all the log parsing, database updates, file sorting/managament. Basically the backend of the system.
The idea is you can have this running on multiple servers, that connect to the website module. Note: This methods will not use ftp.

Website module:
The website the user can interact with that all connected servers send their data to, will include admin control panel.



Would this just be unnecessary, or would it be more convenient?


Notes of what has been done so far:

Log Importer

- Scans the logs directory for .log files, ignoring .tmp files(I can make an options that will allow the admin to import these files, but advanced stats for stats_player events won't be there).
- NUTStats keeps track of all imported files, if the file has already been imported it will be ignored to prevent duplicates.

TODO:
- Let the admin choose what happens to compiled logs, this could be deleting, moving, compressing and moving.

[Darkelarious]

Would this just be unnecessary, or would it be more convenient?

This sounds a lot like the interfaces I wrote for 333networks and the masterserver.

333networks hosts (several) masterserver(s) and stores the servers in a database. Our website can access the database and display server information (close to) real-time.

We also have a jQuery frontend in combination with a json API with our masterserver database.
http://333networks.com/json --> json API documentation

Working example:
http://ubrowser.333networks.com/ut/

Is this the kind of thing you have in mind?

[Shrimp]

It's definitely a better architecture to have a separate backend/processing service, versus a combined website/backend thing.

Depending on how you want to do things, you might even consider breaking the backend into two services as well:

- import/submission/processing: this is where the logs are pushed, parsed, and written to the DB

- view/query/api service: this reads from the DB the processing service writes to, and is what the frontend connects to

Allows you to scale things (har har, not like we need to "scale" UT services for more than 5 hits a week, but whatever, its fun to design these things anyway) separately, and you can also do things like redeploy or improve the parsing stuff without breaking the website, and vice versa.

Some modern tooling would definitely be cool

[UT Sniper (SJA94)]

Would this just be unnecessary, or would it be more convenient?

This sounds a lot like the interfaces I wrote for 333networks and the masterserver.

333networks hosts (several) masterserver(s) and stores the servers in a database. Our website can access the database and display server information (close to) real-time.

We also have a jQuery frontend in combination with a json API with our masterserver database.
http://333networks.com/json --> json API documentation

Working example:
http://ubrowser.333networks.com/ut/

Is this the kind of thing you have in mind?

Yeah that's what I have in mind. It's nice to see someone else who likes to recreate UT look for webpages

Nexgen system for my INI site:



One of many projects I start but never finish..

It's definitely a better architecture to have a separate backend/processing service, versus a combined website/backend thing.

Depending on how you want to do things, you might even consider breaking the backend into two services as well:

- import/submission/processing: this is where the logs are pushed, parsed, and written to the DB

- view/query/api service: this reads from the DB the processing service writes to, and is what the frontend connects to

Allows you to scale things (har har, not like we need to "scale" UT services for more than 5 hits a week, but whatever, its fun to design these things anyway) separately, and you can also do things like redeploy or improve the parsing stuff without breaking the website, and vice versa.

Some modern tooling would definitely be cool

What I was also thinking of doing is was a module that is compatible with the current utstats database, so people can still access old matches from previous versions of utstats.

I also want to make this in such a way other people can create and add their own modules to add support for custom game types if needed, as well as other stuff like I made for the current UTstats like this: https://github.com/scottadkin/UTstats-client-side-screenshots-plugin. What I really want to do is make a match replay system, where a user can watch a match progress second by second with an interactive HTML5 canvas, where users can toggle different scoreboards, server info, and have match events played like sprees, multi kills, flag captures.. and so on.

[iSenSe]

What I really want to do is make a match replay system, where a user can watch a match progress second by second with an interactive HTML5 canvas, where users can toggle different scoreboards, server info, and have match events played like sprees, multi kills, flag captures.. and so on.

That sounds like a dream!

[UT Sniper (SJA94)]

What I really want to do is make a match replay system, where a user can watch a match progress second by second with an interactive HTML5 canvas, where users can toggle different scoreboards, server info, and have match events played like sprees, multi kills, flag captures.. and so on.

That sounds like a dream!

Very early video of what I mean, so far it only does kills... and me being me I forgot to set the player' team during import so I need to do that This is a ctf match, but because I forgot to set the players teams you have to imagine its a team scoreboard. There is no sounds yet or special events.






What I plan on doing is make it so you can cycle through the players via the canvas so you can hear their special events, as well as the match events like flag captures and so on.




-----------------------------------------------------------------------------edit------------------------------------------------




-------------------------------------------------------------------------------edit----------------------------------------------------





-------------------------------------------------------------------------------edit--------------------------------------------------------


Done some more work on the replay system, system now keeps tracks of player connects/disconnects and team changes:








----------------------------------------------------------edit----------------------------------------------------------------------------------

User can now cycle through all the players currently in the game, you will only hear the multi kills and spree events for the player you are currently spectating(I haven't added headshots yet). If another player gets a spree event you will hear the same sound as you do in game when another players gets a spree event.



----------------------------------------------------efit---------------------------------







---------------------------------------------------------edit--------------------------------------------

Early match report page, the players won't be like that colour wise, just did that for testing purposes.

[UT Sniper (SJA94)]

Just had a thought, would people be interested if I made a discord bot for my new utstats system.

Example of what I mean:
.stats player <playername>

<url to player page>
<player has played 237 games>
<last played 23 hours 4 minutes ago>
<link to last played game>

.stats gametype <gametype>

<url to gametype match history>
<2104 game played of <gametype>
<last played 21 minutes 5 seconds ago>
<link to last played game>

Sorry for double post, been editing last post last 3 days, wanted to ask in new post about a discord bot.




--------------------------------edit----------------------------------------

Dm scoreboard, doesn't have FPH and pings yet:

[papercoffee]

Sorry for double post, been editing last post last 3 days

That's ok ... this doesn't count as double post.

[UnrealGGecko]

Just had a thought, would people be interested if I made a discord bot for my new utstats system.

Example of what I mean:
.stats player <playername>

<url to player page>
<player has played 237 games>
<last played 23 hours 4 minutes ago>
<link to last played game>

.stats gametype <gametype>

<url to gametype match history>
<2104 game played of <gametype>
<last played 21 minutes 5 seconds ago>
<link to last played game>

Considering there are some active clans that host servers and hage a Discord channel, I say there might be interest on this. Just gotta let them know.

Heck, I might try to put it in the ut99org Discord as well, if its easy to put in

[UT Sniper (SJA94)]

Just a preview video of a DM game being played at 6 seconds a second. I have made some changes to the server module which will enhance this feature more in the future which will includes pickup logging, which weapon the player killed the other player with, and also added support for smartCTF/DM which will save the players faces/netspeed/FOV which will later be implemented in SmartCTF/DM scoreboards.

The user interacting with the match replay can choose which player they want to view, which will effect what sounds will be played, hud messages, and so on. I haven't implemented pick up hud, you were killed by, as well as timers, and other stats for the gameplay modeimg.





Here is sshot at end of game from NUTStats:


Here is a in game screenshot of the same match, the small difference in Oopers kills is me forgetting to making the player lose a point for suicides, atm it counts suicides as the player has killed someone.









------------------------------------------edit------------------------------

System now automatically imports weapons from log files and save them to database(kill strings):

[UT Sniper (SJA94)]

Match replay now includes item pickups, which includes the sounds for that item when they are picked up. Updated gameplay HUD, as well as some other stuff for the replay system which you can see in this video:




























----------------------------------------edit------------------------------------------

- Added support for Assault.
- Added support for Domination.











--------------------------------edit------------------------

[UT Sniper (SJA94)]

Just a sneak peak of a match replay with the smartCTF scoreboard, also updated the "gameplay" HUD. Just need to add support for LMS and Assault to the match replay(won't take long), then I'll work on the design of the website.



Ignore the fact it's a CTF match with a DM image as the background, haven't added that functionality yet, I need to add import functions for map Images and faces.





------------------------------edit-------------------------------------


Now supports DOM point captures for match replay:



Also added LMS support




Just need to do assault now.

[UT Sniper (SJA94)]

Just a screenshot of a more updated homepage, was doing another project for someone so didn't put much time into nutstats for a while:

[iSenSe]

This looks very awesome, and I am very interested in this for our server!

[UT Sniper (SJA94)]

Just a sshot of a page from top to bottom, I removed stuff via developer tools in browser so you get to see the whole design of a page. There are totals from both all time stats, and records, including more per category then is displayed for single matches:








Callback hell :

Code: Select all

mysql.query(killQuery,(err, result) =>{

            if(err) throw err;

            if(result.length > 0){

                for(let i = 0; i < result.length; i++){
                    this.allTimeRecords.kills.push(result[i]);
                }

               
            }

            mysql.query(deathsQuery, (err, result) =>{
                if(err) throw err;

                if(result.length > 0){
                    for(let i = 0; i < result.length; i++){
                        this.allTimeRecords.deaths.push(result[i]);
                    }
                }

                mysql.query(scoreQuery, (err, result) =>{
                    if(err) throw err;

                    if(result.length > 0){
                        for(let i = 0; i < result.length; i++){

                            this.allTimeRecords.points.push(result[i]);
                        }
                    }

                    mysql.query(suicidesQuery, (err, result) =>{
                        if(err) throw err;
   
                        if(result.length > 0){
                            for(let i = 0; i < result.length; i++){
   
                                this.allTimeRecords.suicides.push(result[i]);
                            }
                        }

                        mysql.query(headshotsQuery, (err, result) =>{
                            if(err) throw err;
       
                            if(result.length > 0){
                                for(let i = 0; i < result.length; i++){
       
                                    this.allTimeRecords.headshots.push(result[i]);
                                }
                            }

                            mysql.query(spreeQuery, (err, result) =>{
                                if(err) throw err;
           
                                if(result.length > 0){
                                    for(let i = 0; i < result.length; i++){
           
                                        this.allTimeRecords.bestSpree.push(result[i]);
                                    }
                                }
                            });
                            mysql.query(multiQuery, (err, result) =>{
                                if(err) throw err;
           
                                if(result.length > 0){
                                    for(let i = 0; i < result.length; i++){
           
                                        this.allTimeRecords.bestMulti.push(result[i]);
                                    }
                                }

                                mysql.query(damageQuery, (err, result) =>{
                                    if(err) throw err;
               
                                    if(result.length > 0){
                                        for(let i = 0; i < result.length; i++){
               
                                            this.allTimeRecords.damage.push(result[i]);
                                        }
                                    }
                                   // this.getMatchRecords();
                                   
                                   // console.log(this.allTimeRecords);
                                });
                            });
                        });
                    });
                });
            });
        });*/

Changed to this:

Code: Select all

//only render the page when all data is loaded
    getResult(query, index, bAllTime){

        let data = [];
        mysql.query(query, (err, result) =>{

            if(err) throw err;
            if(result.length > 0){
                for(let i = 0; i < result.length; i++){
                    data.push(result[i]);
                }
            }

           

            if(bAllTime){
                this.allTimeRecords[index] = data;
                this.currentAllTimeQueries++;
            }else{
                this.matchRecords[index] = data;
            }

            if(this.allTimeQueries == this.currentAllTimeQueries){
                this.res.render("totals",{"allTime":this.allTimeRecords,"match":this.matchRecords});
            }
           
            return data;
        });
    }

Code: Select all


this.getResult(killQuery,"kills", true);
this.getResult(deathsQuery,"deaths", true);
this.getResult(scoreQuery,"points", true);
this.getResult(suicidesQuery,"suicides", true);
this.getResult(headshotsQuery,"headshots", true);
this.getResult(spreeQuery,"bestSpree", true);
this.getResult(multiQuery,"bestMulti", true);
this.getResult(damageQuery,"damage", true);


-------------------edit------------------------------------

Finished player search page