[WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Discussions about UT99
CPan
Novice
Posts: 14
Joined: Sat Mar 10, 2012 5:28 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by CPan » Mon Sep 28, 2020 4:43 am

SOLVED: I did not have a SHOTS directory at the same level as LOGS. I had to create an IMPORTED folder under LOGS as well to keep the LOG Errors from happening since it couldn't move the logs.


Ok so I've given this a go... I have it running on a Pi 4b. I'm at the point to make the node app a service so it'll start automatically. Where I'm really stuck is adding a second FTP server to the config.js in /Importer/api. I have tried a bunch of different comma separated lines and I get an error running the import. It is successful on the first one but fails on the second every time.

Importer/api/Config.js relevant ftpServers block.
IP is the same Windows machine running multiple game servers with IIS pointing the FTP sites to different ports. I can connect to each via FTP (FileZilla) separately but the Importer cannot. Can you nudge me in the right direction to get this to work? It's been over 20 years since I worked with a *nix OS regularly so barney-style will be helpful.

Code: Select all

    ftpServers: [
        {"host": "192.168.x.x", "port": 21, "user": "Username", "password": "pass"},
        {"host": "192.168.x.x", "port": 31, "user": "Username", "password": "pass"}
        //{"host": "127.0.0.1", "port": 21, "user": "scott", "password": "password"},
    ],
Error message

Code: Select all

$ node app
 [22:34:59][Start Up]: --------------------------------------------------
 [22:34:59][Start Up]: --------------------------------------------------
 [22:34:59][Start Up]: ------- Node UTStats Import Module Started -------
 [22:34:59][Start Up]: ---------- Created by Scott Adkin ----------------
 [22:34:59][Start Up]: ---------- Build April 2020 ----------------------
 [22:34:59][Start Up]: ---- To change settings go to /api/config.js -----
 [22:34:59][Start Up]: --------------------------------------------------
New LogParser
┌─────────┬────────────┐
│ (index) │   Values   │
├─────────┼────────────┤
│    0    │ 'Imported' │
└─────────┴────────────┘
┌─────────┐
│ (index) │
├─────────┤
└─────────┘
 [22:34:59][Success]: Connected to 192.168.x.x:21 successfully!
/home/pi/Node-UTStats-master/Importer/api/ftpimporter.js:61
            if(err) throw err;
                    ^

Error: The system cannot find the file specified.
    at makeError (/home/pi/Node-UTStats-master/Importer/node_modules/ftp/lib/connection.js:1067:13)
    at Parser.<anonymous> (/home/pi/Node-UTStats-master/Importer/node_modules/ftp/lib/connection.js:113:25)
    at Parser.emit (events.js:198:13)
    at Parser._write (/home/pi/Node-UTStats-master/Importer/node_modules/ftp/lib/parser.js:59:10)
    at doWrite (_stream_writable.js:415:12)
    at writeOrBuffer (_stream_writable.js:399:5)
    at Parser.Writable.write (_stream_writable.js:299:11)
    at Socket.ondata (/home/pi/Node-UTStats-master/Importer/node_modules/ftp/lib/connection.js:273:20)
    at Socket.emit (events.js:198:13)
    at addChunk (_stream_readable.js:288:12)
Last edited by CPan on Tue Sep 29, 2020 12:51 pm, edited 2 times in total.
CPan

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Mon Sep 28, 2020 6:00 am

Stupid question, have you changed the order of the ftp servers in the config to see if it's always the second one in the list or just the ip:port causing the problems? I've not used IIS before and can't help you in that respect.

CPan
Novice
Posts: 14
Joined: Sat Mar 10, 2012 5:28 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by CPan » Mon Sep 28, 2020 2:58 pm

I just tried it and it connected to port 31 fine but failed on the port 21.


EDIT: Even if I only have one server in the list it still throws the error on line 61 of ftpimporter.js. I don't use ACE on my servers, could that be the problem?
CPan

CPan
Novice
Posts: 14
Joined: Sat Mar 10, 2012 5:28 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by CPan » Mon Sep 28, 2020 11:35 pm

OK. I got it "working". Apparently it fails because I didn't have a SHOTS folder in the UT Directory. Now to troubleshoot why the MH logs that it imported don't show up in there.

Also, I cannot add the server with the same IP in the website. I type in the IP, different port, Optional Friendly name and it just refreshes the screen but doesn't add to the web interface. I can see it adding in the Website terminal that is running though.

Code: Select all

{ mode: 'servers' }
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
{ ip: '192.168.x.x',
  port: '31',
  hostname: 'MonsterHunt',
  type: 'add' }
{ mode: 'servers' }
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
{ ip: '192.168.x.x',
  port: '21',
  hostname: 'SniperServer',
  type: 'add' }
{ mode: 'servers' }
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
MODE = servers
{ ip: '192.168.x.x',
  port: '21',
  hostname: 'SniperServer',
  type: 'add' }
Image of the Admin Panel after the above Website actions.
https://1drv.ms/u/s!AtvY5ZcCa165kpJQIPR ... A?e=7yAK1L
CPan

CPan
Novice
Posts: 14
Joined: Sat Mar 10, 2012 5:28 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by CPan » Wed Sep 30, 2020 4:07 am

OK, I give. It is Importing without errors in the log, moving the files, and updating the log history table in the database BUT nothing is being populated into the database other than that. Every table is empty. I've deleted the log history rows, moved the files back, and re-ran the import and still the same thing. I'm not sure why the import process completes, is able to update the database, but only updates the log history table. This is a MonsterHunt2 Gold server, maybe that's it? Any other ideas?

I've checked DB permissions, the user has GRANT ALL permissions on the database and its tables. Outside of that, I am not sure where to look. Lastly, do you chuckle every time you run a SQL statement @ nut stats?


EDIT: I just put it on my DM Server and it's working right. I guess it has to do with the Game Type for: 0.00 game GameClass MonsterHunt2Gold.MonsterHunt.

Here's my issue in the Importer after I ran it again... Is there a flag to enable checking for MH kills?

Code: Select all

[23:11:49][Warning]: No killdata for this match, skipping. 
 [23:11:49][Success]: Day stats updated. 
 [23:11:49][Success]: Finished importing file [object Object] 
 [23:11:49][Notice]: Starting import for Logs/Unreal.ngLog.2020.09.27.19.36.58.6666.log 
CPan

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Wed Sep 30, 2020 7:47 am

Do you mind sending me one of the logs so I can run some tests so I can fix the problem, I'm guessing it's something to do with a regular expression somewhere, or something stupid like that I messed up somewhere, I'm not at my main computer atm so can't check as easy.


Edit


It's also worth noting if you didn't have the NodeUTStats mutator running at the time of the matches there will be no monster kill data to import, so it won't import anyway.

CPan
Novice
Posts: 14
Joined: Sat Mar 10, 2012 5:28 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by CPan » Wed Sep 30, 2020 11:25 pm

I'm not sure why but it's working now with new logs. I changed the game name from ACE MonsterHunt to just MonsterHunt in the MonsterHunt2Gold INI and it's importing now. I don't see why that would change it since the DM has a space in it too. Once again, problem lies between keyboard and chair.

Thanks for your willingness to look at the logs to see what was going on. I bet that the UTStats wasn't running and the logging was from UT proper.
CPan

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Fri Dec 18, 2020 12:57 am

Just a quick note to say the node utstats project isn't dead, I have been working on it's replacement, the replacement will be one package remade from scratch instead of two separate packages, the website will be made with Next.js which has a lot of advantages to the way I made the site for the previous generation of site which will make adding new features a lot faster. I have also updated the mutator a bit, but not finished it yet(don't use the updated github one with the current generation), you will be able to choose what data is logged and they way some stuff is logged will use less data to store the variables, the importer is backwards compatible with the old/current mutator's behaviour.

Only basic page that I can show atm, most of the stuff I have done is backend related:

Image

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Mon Dec 28, 2020 3:34 am

Just a snap of a players recent matches:
Image

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Thu Jan 07, 2021 8:36 pm

Yay or Nay?

Image

ShaiHulud
Adept
Posts: 416
Joined: Sat Dec 22, 2012 6:37 am

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by ShaiHulud » Fri Jan 08, 2021 1:35 am

I think that looks pretty cool

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Sun Jan 10, 2021 8:43 pm

What i've been toying around with for the top of match pages:

Image



---edit---

After so more tinkering:

Image


-edit--

Image

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Mon Jan 18, 2021 8:14 pm

Just some match page screenshots:

Assault + Match info
Image

Domination
Image

Capture The Flag
Image

Item Pickups
Image

Weapon Stats
Image

Special Events
Image

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Mon Feb 08, 2021 9:54 am

A try at displaying extended ctf cap data:
Image

User avatar
UT Sniper (SJA94)
Masterful
Posts: 645
Joined: Thu Jun 24, 2010 10:35 pm
Personal rank: noob programmer
Location: England

Re: [WIP] Node UTStats Beta 12C (FTP) Nexgen Stats Viewer

Post by UT Sniper (SJA94) » Thu Feb 11, 2021 1:10 pm

Demo video of what the player search page currently will look like:



---edit---
After upload this video I changed it so table headers change sort order to opposite of the current sort order.