2014 site update. Because statistics are beautiful.

Discussion in 'Guild News' started by HittingSmoke, Feb 1, 2014.

2014 site update. Because statistics are beautiful.

Discussion in 'Guild News' started by HittingSmoke, Feb 1, 2014.

  1. HittingSmoke

    HittingSmoke AoA Emeritus

    [FLOAT=right] bB5hXxml.jpg [/FLOAT]Greetings, AoA!

    Before I hop on my boat to the Superbowl (viewing "party" in a small two bedroom apartment) I thought it would be a good time to give an update on how the new web site is holding up, dump some random statistics on you because I love statistics, and tell you about some things that are coming up.

    It's been quite a ride since I signed on with AoA back during the Star Trek Online beta and the face of AoA (our web site) has changed a lot since then. Back then in the before time, in the long long ago, the site was run on a combination of a Joomla CMS front page and phpBB forums. It served everyone well but as we grew it became apparent we needed something more powerful and professional looking. @mynameismonkey entrusted me to take point on an upgrade to vBulletin, which had just come out with a fresh new version that integrated a CMS front page out of the box. Many of us had been using vBulletin 2 and 3 for years with great results. It was one of the most popular commercial forum products on the web. vBulletin 4 would turn out to not live up to the reputation of its predecessors. Slow response times, complete breakages, clunky visual styles, and security breaches became the norm over just the occasional headache. This is the story that's led us up to today, with heaps of analytical data from our new site and nothing but good news to report.

    So far XenForo has been an absolute treat to work with. Our new host is fantastic. All the numbers are in the green and we're running the AoA web backbone with great efficiency. We're paid up on hosting for the next year and we already have the funds in place for the year after with some surplus for other software that may catch our eye. But enough fluff, on to the reason you're really here, the charts!

    The tl;dr of it is our traffic is very steady, our site is extremely responsive, due to server optimizations we use an insignificant fraction of our allotted bandwidth each month, I am a spam-control ninja, and people get to our site with some really fuckin' weird search terms...

    We average just under 150 visitors per day with 9,348 total unique visits since we brought up XenForo. These are mostly unique visits, save for people who log in to the same account on multiple computers. Our data collection is not blind and is quite accurate. There are very few circumstances where a visit from anything other than a browser would be counted:


    Of those 9,348 visits we've had 37,069 unique page views and 51,844 total page views across the site. That's roughly 25,000 page loads per month:


    The first and third most popular search term used on our internal search is "teamspeak". Perhaps we should take a hint and put a link in the nav bar?

    Search entry terms are much harder to track as Google stopped reporting the search term that led to a site in the referrer when logged into your Google account, which most people have these days. There are still some interesting search terms that have made their way into the site statistics:
    • "A man's perfect day" leads to this post as the first result. It looks like that's posted about a thousand other places around the net so I'm not sure how we got to the top of Google for it. So far we've had over twelve people hit our site from that search result.
    • Five people have found AoA by searching for "ju-c air". For the uninitiated, that's a game controller you stick your penis in [SFW].
    • I have no idea what VCXZ is but people keep coming here from Google Images after searching for it.
    • At least two people have gotten here searching "Ping 17ms" and finding this post as a result.
    42 people are filthy heathens who have found our web site by searching Bing.

    5 people need to be sterilized so they can't pass on whatever gene is causing them to use AOL search.

    Neverwinter has been the game to drive the most traffic to AoA through links on other forums. Infestation is the second thanks to whoever linked us on the Infestation clan wiki page. Rift and STO are tied for third.

    [FLOAT=left] download.png [/FLOAT]Chrome is by far the most popular browser among AoAers netting 51% of visits. Firefox is a distant second at 19%. IE trails at 5%, being unable to even beat Mobile Safari or Chrome, coming in at 9% and 8% respectively.

    Mobile use has skyrocketed since we got our responsive site design accounting for roughly 17% of total visits.

    Android is the most popular mobile OS among AoA while Apple is the most popular mobile brand. HTC, Samsung, and Google(Nexus) follow in that order.

    Star Wars: The Old Republic is the most popular division forum by a fair margin with Final Fantasy XIV just behind it. General Discussion comes in at the third most popular forum, which means you all need to start posting in General more instead of just sticking to your own forums. :p[CLEAR][/CLEAR]
    The average time spent browsing the site is just under six minutes per visit:


    The average visitor performs 5.7 page views before leaving:


    I saved the best for last, as it's my favorite and I'm going to elaborate on it a bit. The average page load time across the site is 0.36 seconds. This is the full time it takes to completely render the web page in the browser starting when the web server receives the first request:


    For a dynamic site which is image heavy and has a fairly heavy-weight theme, this is phenomenal. This includes downloading of images, CSS, and javascript, parsing of javascript, database queries, and everything else that goes into a web app this complex. I have no solid metrics from vBulletin but from when I'd have to troubleshoot things and look at the resource load charts I'd say the average fell between one second and ten seconds for complete page load time.

    This comes from our highly optimized server stack. Instead of running whatever LAMP stack is available on our server I custom compile a complete web stack from source including optimizations which benefit us, leaving out modules which we don't use. Our server runs at peak efficiency. There is no waste or bloat. Each server binary in the stack is compiled with the elements we need and no more.

    Our caching system has been a huge success. One of the reasons our performance on the previous host was so abysmal is because there were zero caching layers in place. With the new site we turned on PHP opcode caching and user session caching. This means when you load a page the server doesn't have to parse the PHP code from raw scripts and it doesn't need to retrieve your login session info from disk. Everything is highly optimized to serve what's needed with the lowest latency possible between request and rendering.

    When we first rolled out this new server stack I used it as an opportunity to conduct an experiment. That experiment was the Google Pagespeed module. This module captures each outgoing response from the web server, analyzes it, then automatically applies optimizations where they can be made, caching the result for the next request. The biggest tangible results of this optimization filter are fully optimized images and far fewer requests to the server. All javascript and CSS is served in one file, which for web apps like ours which run addons with their own javascript and style sheets makes a considerable improvement in site performance on the browser end. Since this is done automatically for us we can plug in pretty much anything to the forum without having to worry about decreased performance. The web server handles all of the optimization transparently and serves optimized content from a cache.

    Spam has been completely eliminated. I'm particularly proud of this one. I know many of you haven't seen our new signup page as the majority of you are pre-existing members from before XenForo. If you were to visit the new signup page you'd notice a nice surprise. There is no CAPTCHA. We have completely eliminated CAPTCHA making the signup process far less infuriating while actually managing to eliminate spammers in the process. I've explained in detail the method we use in another posts so I won't go on and on about the details, but we've not had one successful bot registration since moving to XenForo and in the same process we've managed to greatly lower the bar of entry to participate in our community.

    That's the past and present summed up, now on to the future. We have quite a few site-related things in the pipe. One of which is a move to a full-featured WSGI handler instead of a simple PHP CGI handler. This will allow us to create and run forward-facing web apps in languages other than PHP such as Python, Ruby, Lua, or Perl. The idea is that this will allow us more power to do things like plug into third-party APIs like that of EVE, Teamspeak, Gametracker, Steam, etc. It's a long way from anything tangible that you'll see but it's going to greatly expand our available options. This change is going to be rolled out as a tentative week-long test in the coming weeks to compare performance to the current web stack then will be re-evaluated to make sure there are no associated costs we're not prepared for.

    Do you hate IRC? I hate IRC. It's such a clunky technology. No simple integration with other services. Horrible permissions management. Having to auth via a bot. Come to think of it, pretty much everything on IRC beyond simply connecting and changing your name has to be done via a bot because of the limitations of the standard. On top of that, IRC web clients suck. The most popular one is a proxy which routes all of your communication through their servers. I don't like my chat being relayed through a third party. Currently I'm testing XMPP as a replacement for our IRC channel on GameSurge. The first beta version of this will be simple connection to a chat server using your forum username if you connect via the web client. The "release" version rollout will include complete auth against your forum account. Future iterations will include features like multiple channels, integration with XMPP-enabled games like STO, Neverwinter, and Star Citizen, and channel permission management via forum groups. We should see the first XMPP beta within a few weeks which will allow anyone to hop on via the web client or via their own XMPP clients.

    And finally, currently in beta testing and right around the corner is XenForo 1.3. This is the first feature release since we moved to XenForo and the second one since we started testing it a few months back. That we've seen two feature releases since I started testing XenForo speaks volumes about how good of a choice it was for our forum. With 1.3 is going to come features that will allow us to fully move over to SSL for a completely encrypted browsing experience. 1.3 will also bring improvements to custom BBCodes and more third party auth options. We'll be enabling Facebook, Google, and Twitter logins alongside the currently available Steam login.

    I think that's everything there is to say right now. AoA is in a great place. Our tech front is once again stabilized after some embarrassing vBulletin-related issues. We're adding new technology to our backend in an effort to put more features in front of you. We have a site we can be proud to invite people to visit. We're set for a great recruiting push this summer and things are only likely to get better between now and then.

    Stay tuned for updates on new features like the announcement of the XMPP chat rollout.

    Enjoy the Superbowl. Suck it, Denver.

    But before I go for realsies, I'd like to touch on one thing. Our analytics gathering. It seems like a ridiculous amount of data to track. I just want everyone to know that we are not using a third party service for our analytics. We using a product called Piwik. Piwik works much like Google Analytics except instead of sending your data to Google to be parsed, it's all kept internally in our own MySQL database. None of our tracking info ever leaves our server and the most that ever gets made of it are these charts. Everything is kept internal and never shared with outside services. If you're worried about data collection or would like to be omitted from our statistics I can provide a resource for opting out. Since nothing is actually being shared with any third party services I'd hope you all would be open to contributing to our site statistics as it helps not only in learning about our site's popularity but also how well it runs and where performance bottlenecks are. It really is very useful information to us and it helps in a more tangible way than just search metrics.
  2. Reese

    Reese Veteran Member

    Enjoyed the read. I like your enthusiasm.
    HittingSmoke likes this.
  3. jeverian

    jeverian Veteran Member

    Why did I get giddy when I read about the XMPP? What does that say about me? I'm a little scared right now and could use a hug.
    Arcfire, syberghost and HittingSmoke like this.

Share This Page