DNM sales, 2013-2015: "Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem"

"Measuring the Longitudinal Evolution of the Online Anonymous Marketplace Ecosystem", Soska & Christin 2015 (teaser video):

February 2011 saw the emergence of Silk Road, the first successful online anonymous marketplace, in which buyers and sellers could transact with anonymity properties far superior to those available in alternative online or offline means of commerce. Business on Silk Road, primarily involving narcotics trafficking, rapidly boomed, and competitors emerged. At the same time, law enforcement did not sit idle, and eventually managed to shut down Silk Road in October 2013 and arrest its operator. Far from causing the demise of this novel form of commerce, the Silk Road take-down spawned an entire, dynamic, online anonymous marketplace ecosystem, which has continued to evolve to this day. This paper presents a long-term measurement analysis of a large portion of this online anonymous marketplace ecosystem, including 16 different marketplaces, over more than two years (2013-2015). By using long-term measurements, and combining our own data collection with publicly available previous efforts, we offer a detailed understanding of the growth of the online anonymous marketplace ecosystem. We are able to document the evolution of the types of goods being sold, and assess the effect (or lack thereof) of adversarial events, such as law enforcement operations or large-scale frauds, on the overall size of the economy. We also provide insights into how vendors are diversifying and replicating across marketplaces, and how vendor security practices (e.g., PGP adoption) are evolving. These different aspects help us understand how traditional, physical-world criminal activities are developing an online presence, in the same manner traditional commerce diversified online in the 1990s.

...In particular, we are able to measure the effect of the Silk Road takedown on the overall sales volume; how reported "scams" in some marketplaces dented consumer confidence; how vendors are diversifying and replicating across marketplaces; and how security practices (e.g., PGP adoption) are evolving. These different aspects paint what we believe is an accurate picture of how traditional, physicalworld criminal activities are developing an online presence, in the same manner traditional commerce diversified online in the 1990s.

We discover several interesting properties. Our analysis of the sales volumes demonstrates that as a whole the online anonymous marketplace ecosystem appears to be resilient, on the long term, to adverse events such as law enforcement take-downs or "exit scams" in which the operators abscond with the money. We also evidence stability over time in the types of products being sold and purchased: cannabis-, ecstasy- and cocaine-related products consistently account for about 70% of all sales. Analyzing vendor characteristics shows a mix of highly specialized vendors, who focus on a single product, and sellers who sell a large number of different products. We also discover that vendor population has long-tail characteristics: while a few vendors are (or were) highly successful, the vast majority of vendors grossed less than $10,000

...Our measurement methodology consists of 1) crawling online anonymous marketplaces, and 2) parsing them. Table 1 lists all the anonymous marketplaces for which we have data. We scraped 35 different marketplaces a total of 1,908 times yielding a dataset of 3.2 TB in size. The total number of pages obtained from each scrape ranged from 27 to 331,691 pages and performing each scrape took anywhere from minutes up to five days. The sheer size of the data corpus we are considering, as well as other challenging factors (e.g., hidden service latency and poor marketplace availability) led us to devise a custom web scraping framework built on top of Scrapy [3] and Tor [16], which we discuss first. We then highlight how we decide to parse (or ignore) marketplaces, before touching on validation techniques we use to ensure soundness of our analysis.

...We managed to emulate typical browser behavior in all but one case (BlueSky). We were unable to collect meaningful data on BlueSky, as an anti-scraping measure on the server side was to annihilate any session after approximately 100 page requests, and get the user to log in again.

I feel oddly heartened that they weren't able to handle Blue Sky either, although as far as I could tell, the critical count wasn't 100 requests but something else which could trigger well before 100 pages were downloaded.

To ensure that the analysis we performed was not biased, and as a safety against egregious errors, both authors of this paper concurrently and independently developed multiple implementations of the analysis we present in the next section. During that stage of the work, the two authors relied on the same data sources, but used different analysis code and tools and did not communicate with each other until all results were produced. We then internally confirmed that the independent estimations of total market volumes varied by less than 10% at any single point in time, and less than 5% on average, well within expected margin of errors for data indirectly estimated from potentially noisy sources (user feedback). 6

This is really nice.

Our prudent strategy of estimating sales volume from confirmed observations of feedback diverges from other, simpler approaches, such as counting the number of item listings offered (see, e.g., [15]). For instance, over the observed lifetime of Evolution, a few of the most successful item listings had feedback entries that indicated over 1 million dollars had been spent on each of them. The presence of these highly influential item listings suggests that simply counting the total number of listings on a site is a very poor indicator of sales volume. This claim is compounded by the observation that the average sales per item listing per day on Evolution in early July of 2014 was $85.14; but by September 2014, after new vendors and item listings had entered, the sales per item listing had declined to $19.42. Such volatile behavior is particularly common in marketplaces that are small or are going through periods of rapid growth.

Sales over time:

In early 2013, we only have results for Silk Road, which at that point grossed around $300,000/day, far more than previously estimated for 2012 [13]. This number would project to over $100M in a year; combined by the previous $15M estimate [13] for early 2012, and "filling in" gaps for data we do not have in late 2012, appears consistent with the (revised) US Government calculations of $214M of total grossed income by Silk Road over its lifetime, based on Bitcoin transaction logs. These calculations were presented during the trial of the Silk Road founder (evidence GX940).

...Only around late November 2013 did the ecosystem find a bit more stability, as Silk Road 2.0 had been launched and was rapidly growing. In parallel Pandora, Agora, and Evolution were also launched. By late January 2014, volumes far exceeded what was seen prior to the Silk Road take-down. At that point, though, a massive scam on Silk Road 2.0 caused dramatic loss of user confidence, which is evidenced by the rapid decrease until April 2014, before it starts recovering. Competitors however were not affected. (Agora does show spikes due to very imprecise feedback timing at a couple of points.) Eventually, in the Fall of 2014, the anonymous online marketplace ecosystem reached unprecedented highs. We started collecting data from Evolution in July, so it is possible that we miss quite a bit in the early part of 2014, but the overall take-away is unchanged. Finally, in November 2014, Operation Onymous [38] resulted in the take-down of Silk Road 2 and a number of less marketplaces. This did significantly affect total sales, but we immediately see a rebound by people going to Evolution and Agora. We censor the data we obtained from February 2015: at that point we only have results for Agora and Evolution, but coverage is poor, and as explained in Section 3, is likely to underestimate volumes significantly. We did note a short volume decrease prior to the Evolution "exit scam" of March 2015. We have not analyzed data for other smaller marketplaces (e.g., Black Bank, Middle Earth, or Nucleus) but suspect the volumes are much smaller. nally, more recent marketplaces such as AlphaBay seem to have grown rapidly after the Evolution exit scam, but feedback on AlphaBay is not mandatory, and thus cannot be used to reliably estimate sales volumes. In short, the entire ecosystem shows resilience to scams - Sheep, but also Pandora, which, as we can see started off very well before losing ground due to a loss in customer confidence, before shutting down. The effect of law enforcement take-downs (Silk Road 1&2, Operation Onymous) is mixed at best: the ecosystem relatively quickly recovered from the Silk Road shutdown, and appears to have withstood Operation Onymous quite well, since aggregate volumes were back within weeks to more than half what they were prior to Operation Onymous.

Sales by drug category:

We then applied the classifier to the aggregate analysis performed earlier. In addition to placing a particular feedback in time, and pairing it with an item listing observation to derive the price, we predicted the class label of that listing and aggregated the price by class label. Figure 7 shows the normalized market aggregate by category. Drug paraphernalia, weapons, electronics, tobacco, sildenafil, and steroids were collapsed into a category called 'Other' for clarity. Over time the fraction of market share that belongs to each category is relatively stable. However, around October of 2013, December 2013, March 2014, and January 2015, cannabis spikes up to as much as half of the market share. These spikes correspond to the earlier mentioned 1) take-down of Silk Road, 2) closure of Black Market Reloaded and Sheep scam, 3) Silk Road 2.0 theft [5], and 4) Operation Onymous respectively. These are all events that generated substantial doubts in both vendors and consumers regarding the safety and security of operating on these marketplaces. At these times the perceived risk of operation was higher, which may have exerted pressure towards buying and selling cannabis as opposed to other products for which the punishment if caught is much more severe. ...Figure 7 shows that after an event such as a take-down or large scale scam occurs, it takes about 2-3 months before consumer and vendor confidence is restored and the markets converge back to equilibrium. At equilibrium, cannabis and MDMA (ecstasy) are about 25% of market demand each with stimulants closely behind at about 20%. Psychedelics, opioids, and prescription drugs are a little less than 10% of market demand each, although starting in November 2014, prescription drugs have gained significant traction--perhaps making anonymous marketplaces a viable alternative to unlicensed online pharmacies.

Sellers:

The main takeaway from Figure 8 is that the number of sellers overall has considerably increased since the days of Silk Road. By the time Silk Road stopped activities in 2013, it featured around 1,400 sellers; its leading competitors, Atlantis and Black Market Reloaded (BMR) were much smaller. After the Silk Road take-down (October 2013) and Atlantis closure, we observe that both BMR and the Sheep marketplace rapidly pick up a large influx of sellers. In parallel, Silk Road 2.0 also grows at a very rapid pace. Successful newcomers like Pandora, Agora, and Evolution also see quick rises in the number of sellers. After a certain amount of time, however, per-marketplace population tends to stabilize, even in the most popular marketplaces. On the other hand, we also observe that some marketplaces never took off: The Marketplace, Hydra, Deepbay, and Tor Bazaar, for instance, consistently have a small number of vendors. In other words, we see very strong network effects: Either marketplaces manage to get initial traction and then rapidly flourish, or they never manage to take off.

...The key findings here are that half of the sellers are only present for 220 days or less; half of the aliases only exist for 172 days or less. More interesting is the "long-tail" phenomenon we observe: a number (more than 10%) of sellers have been active throughout the entire measurement interval. More generally approximately 25% of all sellers are "in it for the long run," and remain active (with various aliases on various marketplaces) for years. ...About 70% of all sellers never managed to sell more than $1,000 worth of products. Another 18% of sellers were observed to sell between $1,000 and $10,000 but only about 2% of vendors managed to sell more than $100,000. In fact, 35 sellers were observed selling over $1,000,000 worth of product and the top 1% most successful vendors were responsible for 51.5% of all the volume transacted. Some of these sellers, like "SuperTrips" (or to a lesser extent, "Nod") from Silk Road, have been arrested, and numbers released in connection with these arrests are consistent with our findings [4, 6].

PGP key availability:

We then plot in Figure 13 the fraction of vendors, over time, that have (at least) one usable PGP key. We take an extremely inclusive view of PGP deployment here: as long as a vendor has advertised a valid PGP key for one or her active aliases, we consider they are using PGP. As vendors deal with highly sensitive information such as postal delivery addresses of their customers, we would expect close to 100% deployment. We see that, despite improvements, this is not the case. In the original Silk Road, only approximately 2/3 to 3/4 of vendors had a valid PGP key listed. During the upheaval of the 2013 Fall, with many marketplaces opening and shutting down quickly, we see that PGP deployment is very low. When the situation stabilizes in January 2014, we observe an increase in PGP adoption; interestingly, after Operation Onymous, adoption seems even higher, which can be construed as an evolutionary argument: marketplaces that support and encourage PGP use by their sellers (such as Evolution and Agora) might have been also more secure in other respects, and more resilient against takedowns. Shortly before the Evolution shutdown, PGP deployment on Agora and Evolution was close to 90%.

(I would interpret this as a paranoia issue. Operation Onymous was driven by the clustering of DNMs by their operators on a few European hosting services, which could be picked off in batches; what would this have to do with sellers providing PGP keys? Also, if it's a selection/evolutionary difference, the PGP usage should not change in the survivors, rather, PGP usage should predict Onymous survival.)

Also good is cross-checking their estimates against the criminal case documents for Ulbricht & Benthall, and Boosie's leaked seller profile. The consistency of their estimates with those leaks supports the estimates being reasonably accurate despite the many challenges in turning crawls into solid sales numbers.

Graphs:

Media:


Comments


[7 Points] None:

Hey, you're back!


[3 Points] sobulbous:

This whole piece is really interesting but the Seller info is extremely interesting.


[2 Points] Theeconomist1:

Hi there! Glad to see you pop up! Hope you are doing well.


[1 Points] Deafcunt:

Solid info as always.


[1 Points] HofmannsHeir:

Thanks for the good read Gwern!

Can't wait for /u/Vendor_BBMC to come around on some meth rant


[1 Points] None:

TY based gwern