Forum

> > CS2D > Scripts > usgn id > name table
Forums overviewCS2D overview Scripts overviewLog in to reply

English usgn id > name table

25 replies
Page
To the start Previous 1 2 Next To the start

old usgn id > name table

FlooD
GAME BANNED Off Offline

Quote
not sure if someone's already done this but check it out

https://gist.github.com/1130775

https://raw.github.com/gist/1130775/6d86f55e0cbaf0873d63726548eda6b8c512d13c/usgn_name.lua

1
2
3
4
5
6
for i in {0..54900..50}
do
	curl -s -O "http://unrealsoftware.de/users.php?o=1&d=0&il=&f=&p=0&start=$i"
	mv "users.php?o=1&d=0&il=&f=&p=0&start=$i" $(($i/50))
	echo $(($i/50))
done
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(void)
{
	char filename[5];
	int filenum = 0;
	FILE *file;
	char buffer[100000];
	char *ptr;
	char *ptrr;
	puts("usgn_name = {}");
	do {
		sprintf(filename, "%d", filenum);
		file = fopen(filename, "r");
		if (file == NULL) break;
		while (fgets(buffer, 100000, file)) {
			ptr = buffer;
			while (ptr = strstr(ptr, "userid")) {
				ptr += 7;
				ptrr = strchr(ptr, '"');
				*ptrr = 0;
				printf("usgn_name[%s] = ", ptr);
				ptr = ptrr + 2;
				ptrr = strchr(ptr, '<');
				*ptrr = 0;
				printf("\"%s\"\n", ptr);
				ptr = ptrr + 1;
			}
		}
		fclose(file);
	} while (++filenum);
	return 0;
}

old Re: usgn id > name table

Lee
Moderator Off Offline

Quote
You should have it setup as a web service that updates once a day. Just cache the after parameter of the last page scraped and the last US id, scrape that page plus all of the subsequently new pages, and then filter out the US ids that are strictly less than the last US id, and then add that into your database.

Also, bundle lua socket for windows and use curl or wget on *nix distributions to get the latest table.

old Re: usgn id > name table

FlooD
GAME BANNED Off Offline

Quote
user Lee has written
You should have it setup as a web service that updates once a day. Just cache the after parameter of the last page scraped and the last US id, scrape that page plus all of the subsequently new pages, and then filter out the US ids that are strictly less than the last US id, and then add that into your database.

Also, bundle lua socket for windows and use curl or wget on *nix distributions to get the latest table.

eh... nah you can do that

old Re: usgn id > name table

Lee
Moderator Off Offline

Quote
-.-, fine, here you go

last.py
1
state = (71190, 1099, 1312753022)

cron.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
from state import state
import time, urllib, re

p = (re.compile(r"\<td .+?\>(\d+?)\<\/td\>\<td\>\<a .+?\>(.+?)\<\/a\>"), re.compile(r"(\d+)</option></select>"))

def get(page):
	html = urllib.urlopen("http://unrealsoftware.de/users.php?o=1&d=0&il=&f=&p=0&start=%s"%((page-1)*50)).read() # page starts at 1
	return p[0].findall(html), int(p[1].search(html).group(1))

if time.time()-state[2] > 86400:
	page = state[1]
	newstate = state
	f = open("usgn.lua", "a+")
	while True:
		users, last = get(page)
		for id,name in users:
			if int(id)<=state[0]: continue
			f.write('usgn_name[%s] = [[%s]]\n'%(id,name))

		page += 1
		newstate = (int(id), last, time.time())
		if page > last:
			break
	f.close()
	f = open("state.py", "w")
	f.write("state = %s"%repr(newstate))
	f.close()

edit: forgot a parenthesis
edited 1×, last 08.08.11 12:30:38 am

old Re: usgn id > name table

DC
Admin Off Offline

Quote
so.. many.. HTTP requests *explode*
ever thought about actually asking me for a BETTER solution instead of executing tons of queries with loads of overhead?

old Re: usgn id > name table

Lee
Moderator Off Offline

Quote
Well, to be fair, programmers aren't really known for their sane decisions

but I agree, there could be a better solution coupled closer to the native API, on the other hand though, the script only makes 2-5 requests per day so the resource usage is pretty tame.

old Re: usgn id > name table

DC
Admin Off Offline

Quote
yeah that's right. good improvement there.
I just spent 5 mins to make it even better:
http://unrealsoftware.de/users.php?raw&s=0&c=100000000
1
http://unrealsoftware.de/users.php?raw&s=0&c=100000000

params:
raw=request raw format, value doesn't matter
s=start offset (row count, not usgn id!)
c=count

output format:
ID,Name<linebreak>

this will make parsing much easier and faster + reduce the overhead to a minimum.

old Re: usgn id > name table

Lee
Moderator Off Offline

Quote
awesome, no more worries whenever the style changes

state.py
1
last_s = 54900

cron.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
from state import last_s
import urllib

newusers = ["usgn_name[%s] = [[%s]]"%tuple(user.split(",")) for user in urllib.urlopen("http://unrealsoftware.de/users.php?raw&s=%s&c=1000").read().splitlines()]

last_s += len(newusers)

# file IO code, not really important
f = open("usgn.lua", "a+")
f.write("\n".join(newusers))
f.close()
f = open("state.py", "w")
f.write("last_s = %s"%last_s)
f.close()

old Re: usgn id > name table

dizziness
User Off Offline

Quote
user DC has written
yeah that's right. good improvement there.
I just spent 5 mins to make it even better:
http://unrealsoftware.de/users.php?raw&s=0&c=100000000
1
http://unrealsoftware.de/users.php?raw&s=0&c=100000000

params:
raw=request raw format, value doesn't matter
s=start offset (row count, not usgn id!)
c=count

output format:
ID,Name<linebreak>

this will make parsing much easier and faster + reduce the overhead to a minimum.


Dark corner, he mean what? I dosen't understand it

old Re: usgn id > name table

FlooD
GAME BANNED Off Offline

Quote
user Lee has written
Well, to be fair, programmers aren't really known for their sane decisions
nah probably just me...

crap my list doesn't work for people with weird characters in their names lol

oh well

old Re: usgn id > name table

Apache uwu
User Off Offline

Quote
user DC has written
yeah that's right. good improvement there.
I just spent 5 mins to make it even better:
http://unrealsoftware.de/users.php?raw&s=0&c=100000000
1
http://unrealsoftware.de/users.php?raw&s=0&c=100000000

params:
raw=request raw format, value doesn't matter
s=start offset (row count, not usgn id!)
c=count

output format:
ID,Name<linebreak>

this will make parsing much easier and faster + reduce the overhead to a minimum.


DC you liar

Remember this?

thread offtopic My failure of a signature
DC has written
such a table exists only in the database. I'm not going to export it for signatures. lol.

you clearly put too much time and thoughts into your signature...


THERE WAS A LIST OF ALL THE NAMES!

old Re: usgn id > name table

DC
Admin Off Offline

Quote
I didn't lie. I JUST CREATED this website. it was in the database only before. so such a list didn't exist when I said that quoted stuff. it was the truth.

the difference is that you wanted to have something like that for signatures which is really not necessary.

old Re: usgn id > name table

X-Rated
COMMUNITY BANNED Off Offline

Quote
user Apache uwu has written
Yes easier usgn to name >:D

Or you could just put the ID here:
unrealsoftware.de/profile.php?userid=12345
and get the profile and all info...easy dox.

old Re: usgn id > name table

Banaan
User Off Offline

Quote
That's a lot slower (already did that: file cs2d Get USGN Name ). It was already suggested to create a usgn table there so it doesn't have to connect to unrealsoftware for every new player.

usgn_names.lua has written
usgn_name[71190] = "unsc"
usgn_name[71191] = [[Viihne Campese]]


Everything above is quoted in "", everything below uses [[ ]]. Why the change?

old Re: usgn id > name table

EngiN33R
Moderator Off Offline

Quote
user X-Rated has written
user Apache uwu has written
Yes easier usgn to name >:D

Or you could just put the ID here:
unrealsoftware.de/profile.php?userid=12345
and get the profile and all info...easy dox.


Do you even see what's happening here? They're PROGRAMMING, so you better not screw around. Hence your comment was absolutely out of the thread topic.
To the start Previous 1 2 Next To the start
Log in to reply Scripts overviewCS2D overviewForums overview