Approach to DDoS Filter

Sync Blizzards' Armory data with WoWRoster (addon depreciated no longer works see ApiSync)

Moderators: Ulminia, poetter

Approach to DDoS Filter

Postby poetter » Fri Mar 14, 2008 1:31 am

Hi,

in the last svn commit 378 I implemented a global pause and a inter char pause. The defaults are 0.25 secs for global pause and 10 secs for inter char pause. Global pause is the time between two single fetches and inter char pause is the time between two character updates.

With these values i was able to sync my own and one US guild completely without any errors. As the conditions vary from host to host you may need to play a little with these values.

Let me know how it is doing and witch values you tried. Make sure to do a break if you run into the filter.

If you still have problems let me know your values and what kind of host you are using(dedicated, vps, own ip, what so ever).

As I already made some changes for the latest template system, so this build is available from svn only and requires WoWRoster > svn[1722]. With the next nightly build i will pack another archive for non svn users.
Last edited by poetter on Fri Mar 14, 2008 1:33 am, edited 2 times in total.
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Re: Approach to DDoS Filter

Postby Ricomd » Fri Mar 14, 2008 8:22 pm

Can't wait,
No more lazy guildies!

Many thanks, this is probably one of my most used addon's for the roster.

Thanks!!
Ricomd
WR.net Apprentice
WR.net Apprentice
 
Posts: 13
Joined: Wed Aug 16, 2006 12:41 am

Approach to DDoS Filter

Postby mrgrosser » Sun Mar 16, 2008 2:45 pm

I don't understand where to make these changes. I've looked over for these global pauses and inter char pauses. Then again, I don't have 378 and not sure where to get it.
mrgrosser
WR.net Apprentice
WR.net Apprentice
 
Posts: 7
Joined: Wed Mar 12, 2008 12:24 am

Approach to DDoS Filter

Postby Ricomd » Sun Mar 16, 2008 11:41 pm

Is there any word on when a package is aval? I'd normally just grab it from SVN/CVS but it appears that read only isn't for anonymous, or am I just newb'ing it up here?

Thanks!
Ricomd
WR.net Apprentice
WR.net Apprentice
 
Posts: 13
Joined: Wed Aug 16, 2006 12:41 am

Approach to DDoS Filter

Postby moofa » Mon Mar 17, 2008 7:18 am

Hey poetter,

You can use http://www.sunsean.com/wow/ as a generic backup to the actual armory to get class/level/talents, while its minimal its very easy to get this data with cURL and something is better then nothing.

http://www.sunsean.com/wow/?r=Deathwing&n=tragidy
for example.

Right now using the sourceforge armoryclass I have it working with this site, wowarmory and more.

I may just develop a module for roster and release it using keepalives and actual server side cookies to bypass the firewall as I have done in my testing of blizzards filter.

The slowing down you committed to the SVN is good and all, however it will not solve the problem fully, you know most php5 builds can support lynx type browsing via sockets, bypassing curl and using fopen you can then tokenize the data that you are looking for, while its a slightly complex routine I have yet to see the filter come close to blocking access while using lynx.

It's time to take a whole new outlook on the armorysync and start modding the armoryclass as you deem fit, otherwise armorysync will surely fail for anyone on a server with a shared ip that may have other rosters running on the given box.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Approach to DDoS Filter

Postby zanix » Mon Mar 17, 2008 9:24 am

Ricomd: The SVN repository for Roster addons is roster1_addons
Read the Forum Rules, the WiKi, and Search before posting!
WoWRoster v2.1 - SigGen v0.3.3.523 - WoWRosterDF
User avatar
zanix
Admin
Admin
WoWRoster.net Dev Team
WoWRoster.net Dev Team
UA/UU Developer
UA/UU Developer
 
Posts: 5546
Joined: Mon Jul 03, 2006 8:29 am
Location: Idaho Falls, Idaho
Realm: Doomhammer (PvE) - US

Re: Approach to DDoS Filter

Postby Ricomd » Mon Mar 17, 2008 10:54 am

zanix wrote:Ricomd: The SVN repository for Roster addons is roster1_addons


Thanks for the info Zanix, I was able to get the latest no problem now. Although now I'm getting I need to update Roster itself to 1.9.9.1722, but it appears anonymous read only to the roster2 folder isn't there, but I seem to be able to access the folder via WebSVN without issue. Am I just missing something silly here?

Thanks!
Ricomd
WR.net Apprentice
WR.net Apprentice
 
Posts: 13
Joined: Wed Aug 16, 2006 12:41 am

Approach to DDoS Filter

Postby poetter » Mon Mar 17, 2008 1:50 pm

And again. WoWRoster 1.9 beta is in the roster1 repository

@moofa
I will have a look at armoryclass
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Approach to DDoS Filter

Postby poetter » Mon Mar 17, 2008 1:56 pm

@moofa

Which sourceforge project do you mean exactly. I found some.
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Re: Approach to DDoS Filter

Postby moofa » Tue Mar 18, 2008 7:22 pm

poetter wrote:@moofa

Which sourceforge project do you mean exactly. I found some.


phparmory.sourceforge.net

Best Regards
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Re: Approach to DDoS Filter

Postby poetter » Tue Mar 18, 2008 7:45 pm

Well can see something new

Heres the fetch code from phparmory. Almost the same as we use it in function.lib.php. What changes did you make to it?

Code: Select all
  247   /**
  248    * xmlFetch
  249    *
  250    * This function returns the string of characters
  251    * returned from an HTTP GET request to the url
  252    * defined in the url parameter. It is interesting
  253    * to note that although the function is called
  254    * xmlFetch, the returned string may not neccesarily
  255    * be serialized XML data when the function is
  256    * called publicly.
  257    *
  258    * @todo Make the function independent of cURL
  259    *
  260    * @param string      $url         URL of the page to fetch data from
  261    * @param string      $userAgent      The user agent making the GET request
  262    * @param integer   $timeout      The connection timeout in seconds
  263    */
  264    function xmlFetch($url, $userAgent = NULL, $timeout = NULL){
  265
  266       if(($userAgent==NULL)&&($this->userAgent)) $userAgent = $this->userAgent;
  267       if(($timeout==NULL)&&($this->timeout)) $timeout = $this->timeout;
  268
  269       if (function_exists('curl_init')){
  270
  271          $ch = curl_init();
  272          $timeout = $this->timeout;
  273          $userAgent = $this->userAgent;
  274
  275          curl_setopt ($ch, CURLOPT_URL, $url);
  276          curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
  277          curl_setopt ($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
  278          curl_setopt ($ch, CURLOPT_USERAGENT, $userAgent);
  279          curl_setopt ($ch, CURLOPT_HEADER, 0);
  280          curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 1);
  281
  282          $f = curl_exec($ch);
  283
  284          curl_close($ch);
  285
  286       } else {
  287
  288       }
  289
  290       return $f;
  291
  292    }
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Approach to DDoS Filter

Postby moofa » Fri Mar 21, 2008 3:06 am

I personally don't make all of the changes, ( Ejeet Team ) we modified the curl timeouts and added keep alive, we have zero problems using the same ip to access the armory with any other application/call except your current version of armorysync.

Ie; we can call the level/class/server of anyone anytime with a 0.7s delay using AJAX and we can spam this data without caching any data on our server.

You just need to keep the connection alive and change the way your reading tokens, read something else to confirm the page other then the way you are doing it currently and your problem is solved.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Approach to DDoS Filter

Postby poetter » Fri Mar 21, 2008 5:08 am

Don't understand that. Even if I keep the connection alive I get 503s. What do you mean with "read something else"?
Image
User avatar
poetter
Roster AddOn Dev
Roster AddOn Dev
 
Posts: 462
Joined: Sat Jun 30, 2007 9:41 pm
Location: Germany/Hamburg

Approach to DDoS Filter

Postby moofa » Fri Mar 21, 2008 5:33 am

Let's use adding a new guild to the roster while using armory sync, your addon looks for a word to confirm the data this is called a token.

However getting a 503 makes a whole new ball game.

Now assuming your using the same IP that got the 503 simply use lynx / browser of any sort and visit the exact same address, you will be able to access it without a 503, hence leading back to the add-ons methods of data harvesting.

The 503 that you are getting is possibly your code not making full use of cURL's XML reading methods.

Simply make it that this addon demands that the server uses PHP 5.X and make full use of SimpleXML which is packed into PHP5 and is simply amazing for using cURL and XML files.

http://ca3.php.net/simplexml <--- simply a great API.

This will solve your 503 problems which are known bugs in the php 4.X tree due to the lack of support for cURL'ing XML files.

Edit:: added more info.

Check out the below link for using SimpleXML and cURL for both grabbing and outputting

http://developers.pando.com/howto-php-previews
Last edited by moofa on Fri Mar 21, 2008 5:38 am, edited 1 time in total.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Approach to DDoS Filter

Postby moofa » Fri Apr 04, 2008 8:41 am

Have you made any progression on your end poetter or are you discontinuing development Armory Sync?

If you are ceasing to develop this software, may I continue to develop your current work.
Ejeet Networks
Powering Warcraft Guild Sites
Create, Play, Evolve
http://www.ejeet.net
User avatar
moofa
WR.net Apprentice
WR.net Apprentice
 
Posts: 34
Joined: Thu Dec 07, 2006 3:04 am
Location: Canada

Next

Return to ArmorySync - Depreciated

Who is online

Users browsing this forum: No registered users and 1 guest

cron