Approach to DDoS Filter

by **poetter** » Sat Apr 05, 2008 7:44 am

Well, no i didn't make any progress and no i will not discontinue. I'm just not having any idea how to get this problem fixed.

AS is using SimpleXML and cURL. I read the links you posted and can't see any difference. All I see is some standard examples on how to use curl. SimpleXML has nothing to do with it in first place. It takes place when data is already fetched and converts XML to what ever you want.

So if you have a fix for that post some code! Not only I will be thankful.

by **jharvey74** » Tue Apr 15, 2008 10:42 pm

I don't know if Blizzard made some changes to the Armory API or if I just got lucky tweaking my ArmorySynch settings, but it seems to be working much better now. It is able to synchronize with the Armory with no errors. The settings I use:

Synch Cutoff Time - 2
Use Ajax - On
Reload Wait Time - 5
Fetch Method - Be Smart
Fetch Timeout - 4
Fetch Retrys - 2
Update Incomplete Data - Off

Turning the incomplete data update off was the last thing I did. It goes through no errors now, but several of the characters don't synch (from incomplete data). But in successive trys, it will eventually synch them all (usually takes me 3 passes for a guild of about 150 members).

by **jerricho** » Mon Apr 28, 2008 9:52 pm

No it's still not working.

After 5 people get updated it no longer will update anyone whether or not I have incomplete on or not.

I would imagine that until Blizzard allows people to do an XML export from their site, it's just not going to work because they are limiting connections due to so much data mining.

by **Rhoy** » Tue Apr 29, 2008 1:22 am

I've been having pretty good luck lately. I generally try to sync a few times throughout the day. When the sync starts to pull data then everything comes through fine. It's an all or nothing type of situation.

We have 52 members and I get all their info in 1 pass. With the 52 members it takes about 2 hours to get everything updated.

I still get a good chunk of failed first tries but this timing appears to be the sweet spot. Any faster and it starts to get ugly.

Sync cutoff time - 1
Use AJAX - On
Reload wait time - 3
Armory Fetch method - per Page
Armory Fetch timeout - 3
Armory Fetch Retrys - 5
Update incomplete data - On
Skip start page - On

hope this helps!

by **jerricho** » Tue Apr 29, 2008 1:57 am

Still not working. I've tried several times today.

by **jharvey74** » Thu May 01, 2008 10:23 pm

I'm still getting the same results from my end. I never get a complete failure anymore (where it would synch first few then fail OR fail on all of them). There are always several who don't update on each pass, and usually takes 3 or 4 passes to get them all. There is one guy who will not update ever (for incomplete data) and I'm not sure why, but oh well. I would suggest playing with all of the settings until you find a config that works for you (if there is one that will). I think the problem is more involved than just Blizzard's throttling and armorysych's core update process. Multiple people are getting different results and different configurations are working for different people, which indicates that conditions between armorysynch and the armory are heavily affecting it's performance.

Just my $0.02.

by **Samadhi** » Thu Jun 12, 2008 7:50 pm

Am I the only one who is being totally locked out by this filter? I can't get even a single one of my members to update. curl simply refuses to connect.

Roster Warning line 1108: UrlGrabber Error [CURL]: connect() timed out!

Has their been any progress on any front regarding how to get this working again?

by **tuigii** » Thu Jun 12, 2008 8:46 pm

Samadhi wrote:Am I the only one who is being totally locked ....

Common mistake.

DO NOT believe that your side of the Internet communication is perfect - and that Blizzard is filtering.

What about your host (or your ISP) : it has a firewall and it is programmed.
CURL is considered as a "lean and mean" tool, and many hosts just block it "to protect their clients" (yeah, I lov'em....Thanks, Mum...).

Dono how to check it, but you get my point.

PS : Idea : load a Apache/PHP/SQL free package on your PC.
Read how it works : be carefull : one A4 to read ^^
Install Wowroster in it.
Check again.
You'll see.

by **Samadhi** » Thu Jun 12, 2008 9:03 pm

Ah, but I can use the same sort of code with a WoW tooltip mod on my forums if I hit the EU armory instead of the US one. If my host were blocking CURL, that wouldn't be true, right?

by **zanix** » Thu Jun 12, 2008 10:54 pm

They may be blocking the US site (www.wowarmory.com) and not the EU one (eu.wowarmory.com)

by **Samadhi** » Thu Jun 12, 2008 11:10 pm

Ok, fair enough. Thanks.

by **petrakid** » Sat Jun 14, 2008 9:51 pm

Sync cutoff time - 1
Use AJAX - On
Reload wait time - 3
Armory Fetch method - per Page
Armory Fetch timeout - 3
Armory Fetch Retrys - 5
Update incomplete data - On
Skip start page - On

These settings ARE working for me!! The process takes a LONG time, especially when updating gear info, but I haven't gotten a red circle yet, and it has not timed out. I think, until blizzard does something to help us out and improve at their end, we are going to have to expect a lengthy update process.

by **MattM** » Wed Jun 18, 2008 4:12 pm

poetter wrote:Hi,

in the last svn commit 378 I implemented a global pause and a inter char pause. The defaults are 0.25 secs for global pause and 10 secs for inter char pause. Global pause is the time between two single fetches and inter char pause is the time between two character updates.

With these values i was able to sync my own and one US guild completely without any errors. As the conditions vary from host to host you may need to play a little with these values.

Let me know how it is doing and witch values you tried. Make sure to do a break if you run into the filter.

If you still have problems let me know your values and what kind of host you are using(dedicated, vps, own ip, what so ever).

As I already made some changes for the latest template system, so this build is available from svn only and requires WoWRoster > svn[1722]. With the next nightly build i will pack another archive for non svn users.

Changing armorysync_global_pause to 10 from .25 and armorysync_inter_char_pause to 20 from 10 has no noticeable difference and speed, and the DDOS prevention kicks in as usual.

I don't think the while loop/usleep code is working to pause anything.

by **MattM** » Wed Jun 18, 2008 4:21 pm

petrakid wrote:Sync cutoff time - 1
Use AJAX - On
Reload wait time - 3
Armory Fetch method - per Page
Armory Fetch timeout - 3
Armory Fetch Retrys - 5
Update incomplete data - On
Skip start page - On

These settings ARE working for me!! The process takes a LONG time, especially when updating gear info, but I haven't gotten a red circle yet, and it has not timed out. I think, until blizzard does something to help us out and improve at their end, we are going to have to expect a lengthy update process.

AJAX is broken and without it on, I get crapped on:

by **MattM** » Wed Jun 18, 2008 4:30 pm

I may need to write another tool like UU for this.