latenitewithstede at gmail dot com || @StedeUS on Twitter || Subscribe with RSS

Saturday, October 18, 2014

Theorycrafting: Occupational Hazards - The Perils of Crafting Decision-Making in WoD Goldmaking

Crafting has always involved some form of decision-making.  Among the most fundamental questions asked when deciding whether to craft an item for resale: Will I make a profit by crafting & selling this item?  How much profit will I make?  The Market Value (MV) of materials and crafts have long been used to formulate an answer to these most basic questions for enterprising goldmakers.  MV calculations can vary by source, but mostly rely on robust non-parametric measures which resist undue influence from sporadic outliers.  Among all current sources, items are considered by their Item ID (a numeric code), and Market Values are assigned based on the auctioning data available each unique Item ID.

Items which roll random secondary stats will generate items with unique suffixes to their item name.  Their Item IDs, however, are identical.  The result is an MV calculation for a crafted item that considers all such random suffixes as a single item.  In previous expansions, items of this nature were restricted to those of uncommon or rare quality.  They were considered a niche market not useful for endgame content.

In WoD, all epic crafted gear will roll random secondary stats.  The potential for bias in the calculation of MV for these items affects basic assessments of the profitability for these crafts.  This will be demonstrated using examples.  A discussion of the limitations of the current available tools & analytics is presented along with a proposed solution.

The most fundamental mechanic of goldmaking using professions is knowing the costs of creating an item and knowing what the expected revenue from selling that item can be.  The net difference is profit, and profit is the key goal of goldmaking.  The most basic answers to these questions are found by evaluating the current pricing data available from the Auction House (AH).  While the process is straightforward, it can quickly become tedious when goldmakers need such answers for many items.  Evaluating the AH for 10 different items can take some time, but 100 items can be cumbersome beyond the point of feasibility.

To address this inefficiency, goldmakers use add-ons, websites, programs, and spreadsheets, among other tools.  While the dynamics of each of these available options is unique, their approach is common - to present a single value that uniquely summarizes the costs and potential revenue of an item.  This value is generally called the "Market Value" (MV) of an item.

MV is calculated in different ways by different sources.  One source may define it as the 15th percentile of the per item cost of all currently posted auctions.  Another source may consider weighted historical data of some percentile.  All tend to have non-parametric elements in their calculation that makes them relatively immune to sporadic outliers which can potentially bias other measures, such as the average.

Being able to condense 10s, 100s, or 1000s of auctions of data for an item into a single value, makes answering "What is this item worth?" a simpler proposition.  The profit of an item is easily calculated as:

Profit = MV(Crafted Item) - MV(Reagents)

Even this calculation can become tedious if several items are considered, though.  This is where add-ons, websites, and spreadsheets can be used to present the results of the above Profit Calculation for 100s of items at a time.  Visual cues can then be implemented to identify the most profitable paths available to the crafter, quickly narrowing their focus and efforts on the most efficient routes available to generating wealth using professions.

Biased Methods
As mentioned above, the tools available that provide MV calulations are robust to incidental outliers.  While concerted, systemic efforts to poison data and influence prices are capable of thwarting any proper analysis of a market, these efforts generally require an extraordinary amount of time and gold to be maintained.  It is for this reason that MV calculations are often trusted so implicitly, without much second thought.

There exist, however, instances where MV is easily biased beyond the point of reliability.  All currently available tools lookup auctioning data by an item's Item ID - a numeric code.  Items in the game that roll random secondary stats when they are crafted or dropped share a common Item ID, even while they are given a suffix to their name that corresponds to the stats they have (i.e. "of the Landslide" for  Strength / Stamina / Haste / Crit) items .  This is something that most players aren't too concerned about.  Addons that browse and post auctions consider items with different names to be unique.  As a result, buyers do not encounter Item IDs, only item names.

However, addons and websites that contain MV data of items in a database consider all these items with unique name suffixes as a single item.  The MV of "Jasper Ring" considers "... of the Invoker" and "... of the Landslide" as well as all other suffixes.  The distinction is straightforward - items with random enchants are summarized as a group, while items without random enchants are summarized individually - in terms of MV.

Understanding the impact the difference can have requires more discussion.  Up through the Mists of Pandaria (MoP) expansion, only uncommon and rare quality items were able to roll random secondary stats on gear that was not soulbound.  This gear did not present many deliberate market opportunities.  Green jewelry has long been used as fodder for disenchanting.  Blue jewelry would proc at a roughly 10% rate while crafting green jewelry in both MoP & Cataclysm.  These were considered, but mainly as byproducts of shuffling jewelry into enchanting materials.  Indeed, most analytics simply used a single value as a surrogate for the value of all such rare proc jewelry.

This was accepted in the shuffle, because they were a byproduct of a mass production process.  Blue procs were created by easily by the dozen during shuffling sessions.  These were only rare quality; they were not the main reason for shuffling.  So while the inexact nature of the MV calculation for these items was known, it was not concerning.

In WoD, though, this phenomenon is no longer confined to ancillary markets.  In WoD, the number and amount of craftable epic items is enormous - 4 armor types, 8 slots each, 4 trinkets, nearly a dozen weapons, and 6 types of jewelry.  Each of these epic crafts will roll random secondary stats.  Each of these will take nearly a week to create, as they require the use of reagents acquired via daily cooldowns.  There will be no mass production of these.

The markets for these items will be key areas of focus for crafters.  Each crafter will be able to choose how to use their cooldown reagents among several different items to craft - boots, gloves, helms or perhaps wands, staves, or darkmoon cards to make trinkets.

In WoD, reforging is gone and while there are items available that will reroll the stats of crafted gear, their cost is non-trivial.  It stands very much to reason that the actual value of certain items, like weapons - or for example, guns - will vary depending on the stats that the item rolls.  Inevitably there will be stats that are considered "Losers" and stats that are considered "Winners" by the players able to use any given item.

The AH value of a Crit/Haste gun may be far higher (Winner) than that of a Mastery/Versatility gun (Loser). Likely, Winners will be in demand over Losers, and their relative prices will reflect that.  With demand higher among Winners, their prices will likely be higher, and their available quantity lower, than Losers.  

With 5 secondary stats in the game in WoD, that means there are 10 unique combinations of 2-stat rolls (discounting the ability to roll the same stat twice).  They won't all be winners or losers, but it's not hard to imagine situations that illustrate where the current MV calculations introduce bias into the the decision-making process of whether or not to craft this gear.  In the example below, it can be seen that this bias is a result of natural market forces, rather than intentional manipulation or poisoning.

Example 1
Consider the 10 possible combinations of random stats as A, B, C, D, E, F, G, H, I, & J, and the following posted on the AH:

A - 95g
A - 100g
A - 110g
A - 120g
B - 140g
C - 170g
D - 180g
E - 250g
F - 270g
G - 290g
H - 300g
I - 400g
J - 500g

Using the values above, the market value, using the 15th percentile is 100g.  If the MV of our materials was 120g, any goldmaker worth his salt would decide based on the MV of 100g to not craft this item.  Since current MV calculations are considering all of these as 1 item, rather than the unique items they are (which our addons consider them as when we post them, but ironically, not when crafting them), the profitability is underestimated.  As a result of this conservative bias, goldmaking opportunities could easily be missed.

It becomes important, then, to understand, not the overall MV of all these items as a whole, but the MV of each unique item.  If we do that, we have the following:

MV(A) - 95g
MV(B) - 140g
MV(C) - 170g
MV(D) - 180g
MV(E) - 250g
MV(F) - 270g
MV(G) - 290g
MV(H) - 300g
MV(I) - 400g
MV(J) - 500g

These values more accurately represent what a goldmaker could expect to sell each unique variant for.  But evaluating all ten of these for each craft once again becomes unwieldy.  The trouble, though, is that it's inherently difficult to characterize the value a crafter can expect from making an item like this with only a single number.  The median of these MVs is a useful starting point - half of the items that can be made are valued below 260g, while the other half is valued above.  However, it's also very useful to know both the worst-case (minimum) and best case (maximum) scenarios, in addition to the likely case (median).

While current analytics and add-ons offer the use of minimums, maximums, and medians, again - these are only available at the Item ID level, rather than at the level of each unique suffix / combination of random stats.  This is dangerous, as will be shown in the example below.

Example 2
Again, consider the 10 possible combinations of random stats as A, B, C, D, E, F, G, H, I, & J, and the following posted on the AH:

A - 150g
B - 250g
C - 260g
A - 350g
A - 400g
D - 450g
J - 600g
J - 1100g

In the early days of expansion (and more frequently in markets where the quantity available is occasionally reduced to zero) players tend to post items far above their true value.  Also, items that are 'perfect' or 'Best in Slot' are usually posted at inflated prices - where the majority of customers may be willing to wait for prices to fall via undercutting.  The example above demonstrates this.  If all the items above are treated as one, the minimum is 150g, the maximum is 1100g, and the median is 375g.

However, if the MV of each unique suffix is first considered as:

MV(A) - 150g
MV(B) - 250g
MV(D) - 450g
MV(J) - 600g

Then the minimum is 150g, the maximum is 600g, and the median is 350g.  Taken as a whole, the interpretation of these three numbers can be very different.  This is most noticeable in the maximum - a number that many goldmakers would hope to roll (best-case).  But in truth, posting at 1100g is a severe overcut with practically no chance of being bought out before the 600g posting for J.

While it is most common to base crafting go / no-go decisions on a single calculated value for the crafted item and its reagents, it is certainly not unprecedented to temper these decisions with some evaluation of the variability of those values.  The most successful goldmakers have openly discussed using spreadsheets that evaluate MV against historical MVs and their respective measures of standard deviations over time.  This helps them to understand if current values are on the waxing or waning end of natural fluctuations in price.

It would not be a far stretch to imagine the use of minimums, maximums, and medians to arrive at a crafting decision.  However, as demonstrated in Example 2, above, it is critical that the MV of each unique random enchant for an item be first considered, and minimums, maximums, & medians be calculated from those MVs.

It is due to these concerns that current analytics, including websites, add-ons, and spreadsheets, must change to accommodate the proper presentation of pricing information - one that retains fidelity to its intended use.  As of today, the current methods are easily biased by the natural variability in markets where random enchants / stats are commonplace.  In Warlords of Draenor, this will encompass all crafted epic markets.

Until such changes are implemented and made available, it is recommended that goldmakers use full information directly from their AH prior to making a crafting decision regarding the epic WoD crafted gear. It is inadvisable to rely on add-ons to accurately asses this information for you.

No comments:

Post a Comment