Dark Crag and Iron Rock. Historic testing sites of live server patching. It’s always a good sign when one of these two fine communities gets shafted with a few hours of downtime, combined with a likely following of extra special bugs, BUT hopefully getting the target bugs fixed in the first place. Now, granted, these players didn’t exactly sign up for this sort of service and abuse, but it’s good for the rest of us. What I find the most curious is that we don’t have a Live PublicTestServer.

Yes, we do have a PTS before patches and it does get some pretty good populations for focused testing and feedback. Regardless of what Mythic decides to do with the feedback and bug reports, it doesn’t seem to affect these patches before they hit Live servers, let alone all the chaos they cause on Live servers to begin with. Admittedly the PTS is different from Live servers. From what I understand the PTS is a pretty clean and sterile environment that the Devs work in to do all their coding OR it’s the sterile environment they base most of their coding on, and it all gets assembled at the same time. Not real clear there, but it’ll do.

The issue is how patch cycles work. Generally, by the time one patch goes live, there’s already another patch being worked on in a future cycle. This means code is being built upon a patch a full cycle before it’s released. Example, since that confuses me a little. Devs work on patch 1.2, it gets approved for release and begins testing, Devs begin work on 1.2.1 using the 1.2 base, 1.2 finishes PTS and is launched with subsequent hotfixes, (here’s the tricky part) amid hotfixes released to 1.2 Devs must adjust 1.2.1 to comply with said hotfixes at the same schedule as prepping 1.2.1 and getting it approved. Things are missed, 1.2.1 is approved and pushed to PTS, old bugs live again. Devs begin work on 1.3 using the 1.2.1 base as 1.2.1 is PTS’d and launched… hotfixes…rinse wash repeat.

Most of this is standard procedure, but why do old bugs live and die with hotfixes and official patching over and over again? Here we need to classify different Dev’s for what they work on. Thusly, in my mind, we have the Build Devs and Hotfix Devs. I can imagine the mentality between the two already.

Build Devs need to have absolute solid stability in whatever they’re working at the time, they’re on a deadline so sometimes corners need to be cut, and shortcuts taken. If their code is bad, or doesn’t work, under sever scrutiny, it gets rejected, and they just wasted the past 2 months doing nothing of consequence. I see these guys getting buried in Hotfix code that they have to scrape through to find out if anything is going to affect their particular code.

Enter Hotfix Devs. These guys are the rock stars of the coding cubicles. They can whip out code like no ones business, and constantly have new fixes to apply. Unfortunately, they have to work with the Live servers and their field of vision thusly becomes very small. Affecting too much at one time can cause people to report bugs currently being worked on. Regardless, these guys tear through code, and spit out results constantly.

For every Hotfix hastily torn asunder to appease the fans, imagine one little thing breaking that’s entirely unrelated. Fixes made in the past become unraveled to fix a bug in the present. The narrow scope of vision allows Hotfix coders only so much rope as they’re focused on their precious Tracker, and little else. Oh sure, there’s likely to be SOME things they have to check for, but in the reality of working against a clock, and seething hoard of pissed off players holding your paycheck against the wall, some things get slid back the the Build Devs.

Viciously nasty circle of life ain’t it?

This is mostly speculation on my part, though I’ve worked in situations similar to this that ultimately fell apart because of it. It’s said that most development happens in this manner, and all is good. Well maybe, but in your case Mythic, it’s just not good enough. Make a Live PTS, send your Build Devs AND Hotfix Devs there. Hell, you wouldn’t even need to distinguish between the two. Give players a reason to be there, even if it’s just to tear ass through the end game with sweet decked out toons insta-leveled to 40 with templated gear. It’s only going to hold their attention for so long, but the simple act of calling it a Test Server will keep the vast public off of it as they go along playing as intended.

Back to the point. It’s not really fair to treat Dark Crag and Iron Rock as your test dummies. God knows you guys have enough extra servers sitting around. Copy over a live gold version to one of them, throw on some character creation templates, and start converting Devs over to it sooner than later. It’s FINE. We can handle going without some new content for a while, at least we can hope you guys will work out the silly amount of “anniversary” bugs that got released with 1.3.1

-most of this article was derived from speculation on the few tidbits of facts that i’ve absorbed over the past months about how the development process works. I may be wrong, but this is the best representation I can give, and I feel that it’s pretty damn close to accurate.-