UnrealArchive.org

If you have found new Downloads or created something by your own, please announce it here

Re: UnrealArchive.org

Postby noccer » Wed Feb 06, 2019 10:54 pm

Damn, that looks great.
As one of my old friends said, sharing is caring.
I dont know what i find best, the design, the mirrors, the self hosting... simply the mirrors are great. And fine that you do not hotlink files!
How did you get all the mappers and map names in the rows? That was some idea i had in mind ages before but was never would be able to do myself :D
A pity i cannot help with files anymore. However, great work and a good way to go <3
Image

>>You can't steal any ip (v4)adresses, there are exactly 4294967296 of them, and they will still exist when you wrote down all of them, or are stored in a (master)servers database ;)<<
noccer
Adept
 
Posts: 318
Joined: Sun Aug 01, 2010 12:15 pm
Personal rank: Proud Terrorist

Re: UnrealArchive.org

Postby Feralidragon » Wed Feb 06, 2019 11:16 pm

That is really cool, really nice design overall, fast and easy to navigate. :rock:
It still lacks some content, which is to be expected at this stage, but that will certainly come with time and community contributions. :D

I also intend to do something similar, but in a different way and a lot more selective on the actual content to serve, but unfortunately I was unable to advance with it thus far, and for the time being, repositories like this one is something really needed, the more redundancy of these places we have, the less likely it is for anything of worth to be lost forever.

:gj:
User avatar
Feralidragon
Godlike
 
Posts: 5075
Joined: Wed Feb 27, 2008 6:24 pm
Location: Liandri
Personal rank: Work In Progress

Re: UnrealArchive.org

Postby FraGnBraG » Wed Feb 06, 2019 11:19 pm

Good job over all. :tu:

how did you populate the meta-data? I wonder since - for example - there are several maps with "unknown" for the author (including one of mine), and what looks like a "guess" at release date of maps... long ago there was a euro fella who created a monster site that was highly auto-magic (varpu fyi) ... this makes me think of that a bit...

curious... :)
User avatar
FraGnBraG
Inhuman
 
Posts: 798
Joined: Sun Jun 13, 2010 5:13 pm
Location: Canada
Personal rank: Good news everyone!

Re: UnrealArchive.org

Postby Shrimp » Thu Feb 07, 2019 6:57 pm

darksonny wrote:Thank you dude, its a huge effort making this nice collection project! and many of the maps does have some picture to preview it before downloading them. Could be nice (maybe this is a wip process or still unfinished task) all maps could have any pic mainly those funny author who posted a nonrelated preview of his map (pictures like bugsbunny or the logo clan, those are they uselees info that does not helps ut fan to decide wether download or not).


The map pics are extracted automarically directly from the Screenshot property set in the map's LevelInfo. Unfortunately if they didn't put a picture in there, there will be no pic. I can't play 30 000 maps and take screenshots of all of them unfortunately :idea:. Sometimes, if the author put some .jpg or other images in the archive, it will automatically use those too, which is why some maps seem to have a lot of screenshots while others only have one.

papercoffee wrote:I noticed you didn't got all mapping contest packs from UT99.org ...I pinned them to our mapping Discord channel.
https://discordapp.com/channels/2267363 ... 7467059205 (the pin symbol on the right upper corner)
If you prefer those downloads via dropbox PM me and I upload them there.


Thanks. I actually downloaded the individual maps when the contest happened, just hadn't added them yet. You'll note that the lead image for "Map packs" on the side actually contains all the contest entries :rock:. Seemed appropriate.

I've added the pack now, but I haven't looked at how to classify BRUT yet so its not there individually but I'll figure it out.

JackGriffin wrote:I probably need to talk with you about uploading my stuff and what gets uploaded. Over the years I've siteripped everything I could get into and downloaded all the files I ever came in contact with. The things explicitly sent to me to private test I've always placed in a distinct folder so I didn't mistakenly share them but the ripped stuff contains all kinds of unreleased content, beta files, test versions, etc. There's for sure things in there that were not meant to be public but I don't know which is which. I just saved everything.

What are your feelings about files like this? Do you want to just mirror everything and remove stuff if the author gets upset? I can work through it and guess at what should be archived safely but if I do that the community will lose out on a bunch of content that's valuable to learn from as well as history of mod development. I also have a lot of anticheat sources and aimbots, etc. What about those? What about bytehacked files? It can all get a little muddy depending on the level of 'community archive' you want to host.

If you want to discuss this privately just drop me an email. I'll explain more in depth what I have saved. The easiest way to visualize it is that many websites owners are gamers and not web designers. The majority of them think that their files are private when they aren't and someone with a crawler is able to grab all over them. Which I did. A lot. :lol2:


As mentioned, thanks for prompting this whole thing. I really got me thinking, and this is currently the result.

I know what you mean about finding stray stuff :D. I found myself in forgotten university FTP servers probably left over from when people used to play UT there, probably accidental Apache auto-index lists, etc, and was able to find stuff :). I can't complain, it was certainly helpful :mrgreen:.

With regards to the files you have though, I'd be interested in grabbing as much of it as possible and then indexing what I can for the archive. If there are random .uz files, or other things that don't look like packaged/released things, my indexer will pass them over, but slurp up everything else it can identify. Its probably easiest if I could download it form you somehow, I haven't had much luck having people upload to me (too slow). I'll PM you this weekend and we can discuss.

Feralidragon wrote:It still lacks some content, which is to be expected at this stage, but that will certainly come with time and community contributions. :D

I also intend to do something similar, but in a different way and a lot more selective on the actual content to serve, but unfortunately I was unable to advance with it thus far, and for the time being, repositories like this one is something really needed, the more redundancy of these places we have, the less likely it is for anything of worth to be lost forever.


As far as lack of content goes, it's definitely lacking a lot of the more recent stuff (and definitely a lot of old still). I've focused on just scraping as many of the existing repositories as possible, which mostly contain things from up to a few years ago. It seems newer stuff is either "released" directly into caches on servers, or as forum attachments/dropbox uploads, which makes collecting it rather difficult.

Once I get uploads figured out, hopefully things will improve and people can submit their own back-catalogs 8).

When you build your own thing, even if it has better curated or more selective content, feel free to make use of the metadata or steal package reading code, images, file lists, etc if it seems to be of use to you.

FraGnBraG wrote:how did you populate the meta-data? I wonder since - for example - there are several maps with "unknown" for the author (including one of mine), and what looks like a "guess" at release date of maps... long ago there was a euro fella who created a monster site that was highly auto-magic (varpu fyi) ... this makes me think of that a bit...


So, for maps, the authors are pulled directly from the LevelInfo within the maps using my Unreal Engine package reading library (https://github.com/shrimpza/unreal-package-lib). "Unknown" authors typically happen if I can't read the properties form LevelInfo for whatever reason (bugs :noidea: :mrgreen:), or there are no names (seem to be a lot of maps people don't fill out that info). For content that's not maps, where no LevelInfo or dedicated "Author" information is available, I try to look in README files, which generally results in a bit of a mess, but does sometimes get it right. Release dates are definitely guesses - the best approach I found was simply to use the modification date of the content/map/etc. So if someone modified your map, or anything else changed the last modification date (like it was zipped by someone whose system time was wrong), you'll get an inaccurate date.

Please let me know if you spot incorrect authors (just send me links to the content page, and the correct author name), I'll happily correct them :help:. This will definitely help for when I build the "Browse by Authors" page.
ShrimpWorks
Unreal Archive - preserving over 20 years of user-created content for the Unreal series!
User avatar
Shrimp
Average
 
Posts: 30
Joined: Wed Oct 10, 2018 11:15 am
Location: South Africa

Re: UnrealArchive.org

Postby UnrealGGecko » Thu Feb 07, 2019 11:01 pm

I did notice a handful of maps where the author doesn't fill in the LevelInfo>Author tab but instead puts his name on the Screenshot itself.
"NHL Rock The Rink" (PS1) announcer wrote:...And there are no stunt doubles in this game, folks!

My work for UT99: Counter-Strike VP, MaleOne+ & FemaleOne+ voicepacks (NEW!), DM-XC-NaliTreeV2 (from the ut99.org Xmas Contest mappack), my small spec of files at Google Drive

List of console converted maps, models & more!
User avatar
UnrealGGecko
Site Staff
 
Posts: 2149
Joined: Wed Feb 01, 2012 11:26 am
Location: Kaunas, Lithuania
Personal rank: GEx the Gecko

Re: UnrealArchive.org

Postby smeerkat » Sun Feb 10, 2019 6:08 pm

Bookmarked! Should this be a sticky thread (or added to an existing sticky)? 8)
User avatar
smeerkat
Experienced
 
Posts: 129
Joined: Sat Apr 04, 2009 6:00 pm
Location: Canada
Personal rank: DHR

Re: UnrealArchive.org

Postby UnrealGGecko » Sun Feb 10, 2019 8:48 pm

smeerkat wrote:Bookmarked! Should this be a sticky thread (or added to an existing sticky)? 8)

Good point, wash your hands before posting from now on :thuup:
"NHL Rock The Rink" (PS1) announcer wrote:...And there are no stunt doubles in this game, folks!

My work for UT99: Counter-Strike VP, MaleOne+ & FemaleOne+ voicepacks (NEW!), DM-XC-NaliTreeV2 (from the ut99.org Xmas Contest mappack), my small spec of files at Google Drive

List of console converted maps, models & more!
User avatar
UnrealGGecko
Site Staff
 
Posts: 2149
Joined: Wed Feb 01, 2012 11:26 am
Location: Kaunas, Lithuania
Personal rank: GEx the Gecko

Re: UnrealArchive.org

Postby Red_Fist » Wed Feb 13, 2019 7:40 am

I think you have these but maybe not all, http://ut99ut.weebly.com/

And the only thing I can think for the great website you did, is put an arrow for Next page, so you can just click next not only the letters., and one at the bottom of the page as well.
Binary Space Partitioning
Red_Fist
Godlike
 
Posts: 1456
Joined: Sun Oct 05, 2008 3:31 am

Re: UnrealArchive.org

Postby Shrimp » Fri Mar 01, 2019 1:21 pm

Hey all,

Just a quick update; I haven't abandoned this or decided to ignore everyone with content and suggestions, just haven't had much time to get stuck into doing anything meaningful recently.

@Red_Fist, I imported your maps, thanks.

I also added ~800 additional bits and pieces scraped from the big BeyondUnreal dump from Archive.org.

I've been planning out the submission process, and have a sketch of how that'll work, just need to get stuck into building it.

In the realm of things I did get done, these little preview card things with custom descriptions for posting in Discord, Slack, and other places that support Open Graph. Mostly pointless but fun.

Image

Image
ShrimpWorks
Unreal Archive - preserving over 20 years of user-created content for the Unreal series!
User avatar
Shrimp
Average
 
Posts: 30
Joined: Wed Oct 10, 2018 11:15 am
Location: South Africa

Re: UnrealArchive.org

Postby Hellkeeper » Fri Mar 01, 2019 1:39 pm

Shrimp wrote:Image

By the moustache of Stalin, I still have this installed in UT2004 and still use it sometimes, one of my best memories of the good old days. Almost brings a tear to my eye to see Arkon's weapon.
You must construct additional pylons.
User avatar
Hellkeeper
Masterful
 
Posts: 695
Joined: Tue Feb 25, 2014 12:32 pm
Location: France
Personal rank: Soulless Automaton

Re: UnrealArchive.org

Postby Dr.Flay » Sat Mar 02, 2019 5:05 am

Heh, I still have Arkon weapons installed also :tu:

You should probably add the other UT2004 patches
https://liandri.beyondunreal.com/Unreal ... 04#Patches
User avatar
Dr.Flay
Site Staff
 
Posts: 3169
Joined: Thu Aug 04, 2011 9:26 pm
Location: Kernow, UK
Personal rank: Chaos Evangelist

Re: UnrealArchive.org

Postby iSenSe » Sat Mar 02, 2019 1:48 pm

You guys knew https://unrealarchives.com/ right ?
That is not indexed yet.
User avatar
iSenSe
Skilled
 
Posts: 234
Joined: Mon Aug 31, 2015 12:58 pm

Re: UnrealArchive.org

Postby Shrimp » Sun Mar 03, 2019 6:56 am

iSenSe wrote:You guys knew https://unrealarchives.com/ right ?
That is not indexed yet.


unrealarchives.com serves a somewhat different and more focused purpose - primarily a somewhat more hand-curated collection of single player content for Unreal and UT. I also debated long and hard on my choice of name, being so closely related to that existing site.

Dr.Flay wrote:You should probably add the other UT2004 patches


I think the only one missing is the x64 patch, which I'll add. You'll see on the patch page that all variations are listed for download, rather than a separate page for each. I've not included the specialised hardware patches since those devices aren't available any more, but I'll work on updating the contribution guide so someone can add them if they feel there's value in having them.
ShrimpWorks
Unreal Archive - preserving over 20 years of user-created content for the Unreal series!
User avatar
Shrimp
Average
 
Posts: 30
Joined: Wed Oct 10, 2018 11:15 am
Location: South Africa

Re: UnrealArchive.org

Postby Skillz » Fri Apr 05, 2019 1:44 am

I am so glad someone finally made something like this. Kudos Shrimp. Nice freaking work man.
Skillz
Novice
 
Posts: 5
Joined: Sun Dec 16, 2018 9:32 pm

Re: UnrealArchive.org

Postby Dr.Flay » Fri Apr 05, 2019 4:19 am

Worth noting for all, GameFront have now added a re-upload feature on pages with missing files.
They must be the same name and size. No idea if they still have hash info for the lost files (doubt it).
They also provide a contact feature where you can give the admin a link to a valid copy, so they can do the upload.
User avatar
Dr.Flay
Site Staff
 
Posts: 3169
Joined: Thu Aug 04, 2011 9:26 pm
Location: Kernow, UK
Personal rank: Chaos Evangelist

PreviousNext

Return to Report new Downloads

Who is online

Users browsing this forum: No registered users and 1 guest