Flash Boys: A Wall Street Revolt

2020-03-05

Michael Lewis

2017-07-18

Introduction: Windows on the World

Chapter 1: Hidden in Plain Sight

What [Dan] Spivey had realized, by 2008, was that there was a big difference between the trading speed that was available between these exchanges and the trading speed that was theoretically possible. Given the speed of light in fiber, it should have been possible for a trader who needed to trade in both places at once to send his order from Chicago to New York and back in roughly 12 milliseconds, or roughly a tenth of the time it takes you to blink your eyes, if you blink as fast you as you can. (A millisecond is one thousandth of a second.) The routes offered by the various telecom carriers–Verizon, AT&T, Level 3, and so on–were slower than that, and inconsistent. One day it took them 17 milliseconds to send an order to both data centers; the next, it took them 16 milliseconds. By accident, some traders had stumbled across a route controlled by Verizon that took 14.65 milliseconds. “The Gold Route,” the traders called it, because on the occasions you happened to find yourself on it you were the first to exploit the discrepancies between prices in Chicago and prices in New York. Incredibly to Spivey, the telecom carriers were not set up to understand the new demand for speed. Not only did Verizon fail to see that it could sell its special route to traders for a fortune; Verizon didn’t even seem aware it owned anything of special value. “You would have to order up several lines and hope that you got it,” says Spivey. “They didn’t know what they had.” As late as 2008, major telecom carriers were unaware that the financial markets had changed, radically, the value of a millisecond.
Spivey had persuaded Jim Barksdale, the former CEO of Netscape Communications […] to fund what Spivey estimated to be a #400 million tunnel. They named the company Spread Networks.
[They built Spread Networks before lining up even a single prospective client, and when they began sales:] Exactly why speed was so important to them was not clear; what was clear was that they felt threatened by this faster new line. “Somebody would say, ‘Wait a second,'” recalls Carley. “‘If we want to continue with the strategies we are currently running, we have to be on this line. We have no choice but to pay whatever you’re asking. And you’re going to go from my office to talk to all of my competitors.'”
The [banks other than Citigroup] all grasped the point of the line but were given pause by the contract Spread required them to sign. This contract prohibited anyone who leased the line from allowing others to use it. Any big bank that leased a place on the line could use it for its own proprietary trading but was forbidden from sharing it with its brokerage customers. To Spread this seemed an obvious restriction: The line was more valuable the fewer people that had access to it. The whole point of the line was to create inside the public markets a private space, accessible only to those willing to pay the tens of millions of dollars in entry fees. “Credit Suisse was outraged,” says a Spread employee who negotiated with the big Wall Street banks. “They said, ‘You’re enabling people to screw their customers.'” The employee tried to argue that this was not true–that it was more complicated than that–but in the end Credit Suisse refused to sign the contract. Morgan Stanley, on the other hand, came back to Spread and said, We need you to change the language. “We say, ‘But you’re okay with the restrictions?’ And they say, ‘Absolutely, this is totally about optics.’ We had to wordsmith it so they had plausible deniability.” Morgan Stanley wanted to be able to trade for itself in a way it could not trade for its customers; it just didn’t want to seem as if it wanted to.

Chapter 2: Brad’s Problem

[Brad Katsuyama’s] troubles began at the end of 2006, after RBC paid $100 million for a U.S. electronic stock market trading form called Carlin Financial [founded by Jeremy Frommer].
As it happened, at almost exactly the moment Jeremy Frommer fully entered Brad’s life, the U.S. stock market began to behave oddly.
Brad’s main role as a trader was to sit between investors who wanted to buy and sell big amounts of stock and the public markets, where the volumes were smaller.
By June 2007 the problem had grown too big to ignore. An electronics company in Singapore called Flextronics announced its intention to buy a smaller rival, Solectron, for a bit less than $4 a share. A big investor called Brad and said he wanted to sell 5 million shares of Solectron. The public stock markets–the New York Stock Exchange (NYSE) and Nasdaq–showed the current market. Say it was 3.70-3.75, which is to say you could sell Solectron for $3.70 a share or buy it for $3.75. The problem was that, at those prices, only a million shares were bid for and offered. The big investor who wished to sell 5 million shares of Solectron called Brad because he wanted Brad to take the risk on the other 4 million shares. And so Brad bought the shares at $3.65, slightly below the priced quoted in the public markets. But when he turned to the public markets–the markets on his trading screens–the share price instantly moved. Almost as if the market had read his mind. Instead of selling a million shares at $3.70, as he’d assumed he could do, he sold a few hundred thousand and triggered a minicollapse in the price of Solectron. It was as if someone knew what he was trying to do and was reacting to his desire to sell before he had fully expressed it. By the time he was done selling all 5 million shares, at prices far below $3.70, he had lost a small fortune.
Finally he complained so loudly that they sent him the developers, the guys who had come to RBC in the Carlin acquisition. […] They were called “The Golden Goose.”
They, too, explained to Brad that he, and not his machine, was the problem. “They told me it was because I was in New York and the markets were in New Jersey and my market data was slow. Then they said that it was all caused by the fact that there are thousands of people trading in the market. They’d say, ‘You aren’t the only one trying to do what you’re trying to do. There’s other events. There’s news.'”
If that was the case, he asked them, why did the market in any given stock dry up only when he was trying to trade in it?
In 2002, [the exchanges] charged every Wall Street broker who submitted a stock market order the same simple fixed commission per share traded. Replacing people with machines enabled the markets to become not just faster but more complicated. The exchanges rolled out an incredibly complicated system of fees and kickbacks. The system was called the “maker-taker model” and, like a lot of Wall Street creations, was understood by almost no one. Even professional investors’ eyes glazed over when Brad tried to explain it to them. “It was the one thing I’d skip, because a lot of people just didn’t get it,” he said. Say you wanted to buy shares in Apple, and the market in Apple was 400-400.05. If you simply went in and bought the shares at $400.05, you were said to be “crossing the spread.” The trader who crossed the spread was classified as the “taker.” If you instead rested your order to buy Apple at $400, and someone came along and sold the shares to you at $400, you were designated a “maker.” In general, the exchanges charged takers a few pennies a share, paid makers somewhat less, and pocketed the difference–on the dubious theory that whoever resisted the urge to cross the spread was performing some kind of service. But there were exceptions. For instance, the BATS exchange, in Weehawken, New Jersey, perversely paid takers and charged makers.
Dark pools were another rogue spawn of the new financial marketplace. Private stock exchanges, run by the big brokers, they were not required to reveal to the public what happened inside them. They reported any trade they executed, but they did so with sufficient delay that it was impossible to know exactly what was happening in the broader market at the moment the trade occurred. Their internal rules were a mystery, and only the broker who ran a dark pool knew for sure whose buy and sell orders were allowed inside. The amazing idea the big Wall Street banks had sold to big investors was that transparency was their enemy. If, say, Fidelity wanted to sell a million shares of Microsoft Corp.–so the argument ran–they were better off putting them into a dark pool run by, say Credit Suisse than going directly to the public exchanges. On the public exchanges, everyone would notice a big seller had entered the market, and the market price of Microsoft would plunge. Inside a dark pool, no one but the broker who ran it had any idea what was happening.
The cost of RBC’s creating and running its own dark pool, Brad now learned, would be nearly $4 million a year. Thus his second question for the Golden Goose: How will we make more than $4 million from our own dark pool? The Golden Goose explained that they’d save all sorts of money in fees they paid to the public exchanges–by putting together buyers and sellers of the same stocks who came to RBC at the same time. If RBC had some investor who wanted to buy a million shares of Microsoft, and another who wanted to sell a million shares of Microsoft, they could simply pair them off in the dark pool rather than pay Nasdaq or the New York Stock Exchange to do it. In theory this made sense; in practice, not so much. “The problem,” said Brad, “was RBC was two percent of the market. I asked how often we were likely to have buyers and sellers to cross. NO one had done the analysis.” The analysis, once finished, showed that RBC, if it opened a dark pool and routed all its clients’ orders into it first, would save about $200,000 a year in exchange fees.
What he had […] was a fast-growing pile of unanswered questions. Why, between the dark pools and the public exchanges, were there nearly sixty different places, most of them in New Jersey, where you could buy any listed stock? Why did the public exchanges fiddle with their own pricing so often–and why did you get paid by one exchange to do exactly the same thing for which another exchange might charge you? How did a firm he’d never heard of–Getco–trade 10 percent of the entire volume of the stock market? How had this guy in the middle of nowhere–in retail in Canada–learned of Getco’s existence before him? Why was the market displayed on Wall Street trading screens an illusion?
In May 2009, […] New York senator Charles Schumer wrote a letter to the SEC […] condemning the stock exchanges for allowing “sophisticated high-frequency traders to gain access to trading information before it is sent out widely to other traders.” […] That was the first time Brad had heard the term “flash orders.” To the growing list of mental questions, he added another: Why would stock exchanges have allowed flash trading in the first place?
[Brad Katsuyama and Rob Park put together a team to figure out the cause of the trading screen problem, and implemented a tool that would compensate for distance-based latency to ensure that orders arrive at all exchanges at the same time.] The tool was always just Thor.
Miek Gitlin, who oversaw $700 billion in U.S. stock market investments for T. Rowe Price. […] what Brad described was a far more detailed picture of the market than Gitlin had ever considered–and, in that market, all the incentives were screwed up. The Wall Street brokerage firm deciding where to send T. Rowe Price’s buy and sell orders had a great deal of power over how and where those orders got submitted. The firms were now paid for sending orders to some exchanges and billed for sending orders to others. Did the broker resist these incentives when they didn’t align with the interests of the investors he was meant to represent? No one could say. Another wacky incentive was called “payment for order flow.” As of 2010, every American stockbroker and all the online brokers effectively auctioned their customers’ stock market orders. The online broker TD Ameritrade, for example, was paid hundreds of millions of dollars each year to send their orders to a high-frequency trading firm called Citadel, which executed the orders on their behalf. Why was Citadel willing to pay so much to see the flow? No one could say with certainty.
the stock market was no longer a market. It was a collection of small markets scattered across New Jersey and lower Manhattan. When bids and offers for shares sent to these places arrived at precisely the same moment, the markets acted as markets should. If they arrived even a millisecond apart, the market vanished, and all bets were off. Brad knew that he was being front-run–that some other trader was, in effect, noticing his demand for stock on one exchange and buying it on others in anticipation of selling it to him at a higher price. He’d identified a suspect: high-frequency traders. “I had the sense that the problems are being caused by this new participate in the market,” said Brad. “I just didn’t know how they were doing it.”
By late 2009 U.S. high-frequency trading firms were flying to Toronto with offers to pay Canadian banks to expose their customers to high-frequency traders. Earlier that year, one of RBC’s competitors, the Canadian Imperial Bank of Commerce (CIBC), had sublet its license on the Toronto Stock Exchange to several high-frequency trading firms and, within a few months, had seen its historically stable 6-7 percent share of Canadian stock market trading triple.
The rules of the Canadian stock market are different from the rules of the U.S. stock market. One rule in Canada that does not exist in the United States is “broker priority.” The idea is to enable brokerage firms that have both sides of a trade to pair off buyers and sellers without the interference of other buyers and sellers. For example, imagine that CIBC (representing some investor) has a standing order to buy shares in Company X at $20 a share, but that it is not alone, and several other banks also have standing orders for Company X’s shares at $20. If CIBC then enters the market with an order from another CIBC customer to sell shares in Company X at $20, the CIBC buyer has priority on the trade and is the first to have his order filled. By allowing high-frequency traders to operate with CIBC’s license, CIBC was, in effect, creating lots of collisions between its own customers and the HFT firms.

Chapter 3: Ronan Ryan’s Problem

Latency was simply the time between the moment a signal was sent and when it was received. There were several factors that determined the latency of a stock market trading system: the boxes, the logic, and the lines. The boxes were the machinery the signals passed through on their way from Point A to Point B: the computer servers and signal amplifiers and switches. The logic was the software, the code instructions that operated the boxes. Ronan didn’t know much about software, except that, more and more, it seemed to be written by Russian guys who barely spoke English. The lines were the glass fiber-optic cables that carried the information from one box to another. The single biggest determinant of speed was the length of the fiber, or the distance the signal needed to travel to get from Point A to Point B. Ronan didn’t know what a millisecond was, but he understood the problem with this Kansas City hedge fund: It was in Kansas City. Light in a vacuum traveled at 186,000 miles per second, or, put another way, 186 miles a millisecond. Light inside of fiber bounced off the walls and so traveled at only about two-thirds of its theoretical speed. But it was still fast. The biggest enemy of the speed of a signal was the distance the signal needed to travel.
Ronan Ryan moved the computers from Kansas City to Radianz’s data center in Nutley and reduced the time it took them to find out what they had bought and sold from 43 milliseconds to 3.8 milliseconds.
As all his new customers housed their computers inside the Radianz data center in Nutley, this was a tricky business. Ronan says, “One day a trader calls me and asks, ‘Where am I in the room?’ I’m thinking, In the room? What do you mean ‘in the room’? What the guy meant, it turned out, was in the room.” He was willing to pay to move his computer that sent the orders into the stock market as close as possible to the pipe that exited the building in Nutley–so that he would have a slight jump on the other computers in the room. Another trader then called Ronan to say that he had noticed that his fiber-optic cable was a few yards longer than it needed to be. Instead of having it wind around the outside of the room with everyone else’s cable–which helped to reduce the heat in the room–the trader wanted his cable to hew a straight line right across the middle of the room.
It was only a matter of time before the stock exchanges figured out that, if people were willing to spend hundreds of thousands of dollars to move their machines around inside some remote data center just so they might be a tiny bit closer to the stock exchange, they’d pay millions to be inside the stock exchange. Ronan followed them there. He came up with an idea: sell proximity to Wall Street as a service. Call it “proximity services.” “We tried to trademark proximity, but you can’t because it’s a word,” he said. What he wanted to call proximity soon became known as “co-location,” and Ronan became the world’s authority on the subject. When they ran out of ways to reduce the length of their cable, they began to focus on the devices on either end of the cable. Data switches, for instance. The difference between fast data switches and slow ones was measured in microseconds (millionths of a second), but microseconds were now critical. “One guy says to me, ‘It doesn’t matter if I’m one second slower or one microsecond; either way I come in second place.'” The switching times fell from 150 microseconds to 1.2 microseconds per trade. “And then,” says Ronan, “they started to ask, ‘What kind of glass are you using?'” All optical fibers were not created equal; some kinds of glass conveyed light signals more efficiently than others. And Ronan thought: Never before in human history have people gone to so much trouble and spent so much money to gain so little speed. “People were measuring the lengths of their cables to the foot inside the exchanges. People were buying these servers and chucking them out six months later. For microseconds.”
One HFT firm he set up inside one of the stock exchanges insisted that he wrap their new computer servers in wire gauze–to prevent anyone from seeing their blinking lights or improvements in their hardware. Another HFT firm secured the computer cage nearest the exchange’s matching engine–the computer code that, in effect, was now the stock market. Formerly owned by Toys “R” Us (the computers probably ran the store’s website), the cage was emblazoned with store logos. The HFT firm insisted on leaving the Toys “R” Us logos in place so that no one would know they had improved their position, in relation to the matching engine, by several feet. “They were all paranoid,” said Ronan. “But they were right to be. If you know how to pickpocket someone and you were the pickpocketer, you would do the same thing. You’d see someone find a new switch that was 3 microseconds faster, and in two weeks everyone in the data center would have the same switch.”
He was […] keenly aware that he had only the faintest idea of the reason for this incredible new lust for speed. He heard a lot of loose talk about “arbitrage,” but what, exactly, was being arbitraged, and why did it need to be done so fast? “I felt like the getaway driver,” he said. “Each time, it was like, ‘Drive faster! Drive faster!’ Then it was like, ‘Get rid of the airbags!’ Then it was, ‘Get rid of the fucking seats!’ Towards the end I’m like, ‘Excuse me, sirs, but what are you doing in the bank?'”
By early 2008 Ronan was spending a lot of his time abroad, helping high-frequency traders exploit the Americanization of foreign stock markets. A pattern emerged: A country in which the stock market had always traded on a single exchange–Canada, Australia, the UK–would, in the name of free-market competition, permit the creation of a new exchange. The new exchange was always located at some surprising distance from the original exchange.
Each new exchange gave rise to the need for high-speed routes between the exchanges. “It was almost like they picked places to set up exchanges so that the market would fragment,” said Ronan.
[Brad interviewed Ronan about HFT.] “What he said told me that we needed to care about microseconds and nanoseconds,” said Brad. The U.S. stock market was now a class system, rooted in speed, of haves and have-nots. The haves paid for nanoseconds; the have-nots had no idea that a nanosecond had value. The haves enjoyed a perfect view of the market; the have-nots never saw the market at all. What had once been the world’s most public, most democratic, financial market had become, in spirit, something more like a private viewing of a stolen work of art.
[Brad hired Ronan and] started to teach Ronan the language of trading. A “bid” was an attempt to buy stock, an “offer” an attempt to sell it. To cross the spread, if you were selling, meant to accept the bidder’s price, or, if you were buying, the offering price.
The fastest any high-speed trader’s signal could travel from the first exchange it reached to the next one was 465 microseconds. […] That is, for Brad’s trading orders to interact with the market as displayed on his trading screens, they needed to arrive at all the exchanges within a 465-microsecond window. The only way to do that, Ronan told his new colleagues at RBC, was to build and control your own fiber network.
[Ronan had maps of where the telecoms used fiber between the exchanges.] To Brad the maps explained, among other things, why the market on BATS had proved so accurate. The reason they were always able to buy or sell 100 percent of the shares listed on BATS was that BATS was always the first stock market to receive their orders. News of their buying and selling hadn’t had time to spread throughout the marketplace.
“I was like, ‘Holy shit, BATS is just closest to us.’ It’s right outside the freaking tunnel.” Inside BATS, high-frequency trading firms were waiting for news that they could use to trade on the other exchanges. They obtained that news by placing very small bids and offers, typically for 100 shares, for every listed stock. Having gleaned that there was a buyer or seller of Company X’s shares, they would race ahead to the other exchanges and buy or sell accordingly.
The orders resting on BATS were typically just the 100-share minimum required for an order to be at the front of any price queue, as their only purpose was to tease information out of investors. The HFT firms posted these tiny orders on BATS–orders to buy or sell 100 shares of basically every stock traded in the U.S. market–not because they actually wanted to buy and sell the stocks but because they wanted to find out what investors wanted to buy and sell before they did it. BATS, unsurprisingly, had been created by high-frequency traders.
a lot of what Ronan had seen and heard didn’t make sense to him: He didn’t know what he knew. Brad now helped him to understand. For instance, Ronan had noticed the HFT guys creating elaborate tables of the time, measured in microseconds, it took for stock market order to travel from any given brokerage house to each of the exchanges. “Latency tables,” these were called. The times were subtly different for every brokerage house–they depended upon where the brokerage house physically was located and which fiber networks it leased in New Jersey. These tables took trouble to create and were of obvious value to high-frequency traders, but Ronan had no idea why. This was the first Brad had heard about latency tables, but he knew exactly why they had been created: They enabled high-frequency traders to identify brokers by the time their orders took to travel from one exchange to the other. Once you had figured out which broker was behind any given stock market order, you could discern patterns in each broker’s behavior. If you knew which broker had just come into the market with an order to buy 1,000 shares of IBM, you might further guess whether those 1,000 shares were the entire order or a part of a much larger order. You might also guess how the broker might distribute the order among the various exchanges and how much above the current market price for IBM shares the broker might be willing to pay. The HFT guys didn’t need perfect information to make riskless profits; they only needed to skew the odds systematically in their favor. But, as Brad put it, “What you’re looking for ultimately is large brokers who are behaving idiotically with their customers’ orders. That’s the real gold mind.”
He also knew that Wall Street brokers had a new incentive to behave idiotically, because he had himself succumbed to the temptation. When Wall Street decided where to route their clients’ stock market orders, they were now greatly influenced by the new system of kickbacks paid and fees charged to them by the exchanges: If a big Wall Street broker stood to be paid to send an order to buy 10,0000 shares of IBM to BATS but was charged to send the same order to the New York Stock Exchange, it would program its routers to send the customer’s order to BATS. The router, designed by human beings, took on a life of its own.
Along with the trading algorithms, the routers were a critical piece of technology in the automated stock markets. Both are designed and built by people who work for the Wall Street broker. Both do the thinking that people used to do, but the intellectual tasks they perform are different. The algorithm does its thinking first: It decides how to slice up any given order. Say you want to buy 100,000 shares of XYZ Company at no more than $25 a share, when the market shows a total of 2,000 shares at $25. To simply attempt to buy 100,000 shares all at once would create havoc in the market and drive the price higher. The algorithm decides how many shares you buy, when to buy them, and the price to pay. For example, it may instruct the router to carve the 100,000-share order into twenty pieces, and to buy 5,000 shares every five minutes, so long as the price is no higher than $25.
The router determines where the order is sent. For instance, a router might instruct the order to go first to a Wall Street firm’s dark pool before going to the exchanges. Or it might instruct the order to go first to any exchange that will pay the broker to trade, and only then to exchanges on which the broker will be compelled to pay to trade. (This is a so-called sequential cost-effective router.) To illustrate how stupid routing can be, say you have told your Wall Street broker–to whom you are paying a commission–that you wish to buy 100,000 shares of Company XYZ at $25 and now, conveniently, there are 100,000 shares for sale at $25, 10,000 on each of ten different exchanges, all of which will charge the broker to trade on your behalf (though far less than the commission you have paid to him). There are, however, another 100 shares for sale, also at $25, on the BATS exchange–which will pay the broker for the trade. The sequential cost-effective router will go first to BATS and buy the 100 shares–and cause the other 100,000 shares to vanish into the paws of high-frequency traders in the bargain relieving the broker of the obligation to pay to trade). The high-frequency traders can then turn around and sell the shares of Company XYZ at a higher price, or hold onto the shares for a few seconds more, while you, the investor, chase Company XYZ’s shares even higher. In either case, the result is unappealing to the original buyer of Company XYZ’s shares.
Then came the so-called flash crash. At 2:45 on May 6, 2010, for no obvious reason, the market fell six hundred points in a few minutes. […] Shares of Procter & Gamble, for instance, traded as low as a penny and as high as $100,000. Twenty thousand different trades happened at stock prices more than 60 percent removed from the prices of those stocks just moments before. Five months later, the SEC published a report blaming the entire fiasco on a single large sell order, of stock market futures contracts, mistakenly place don an exchange in Chicago by an obscure Kansas City mutual fund.
That explanation could only be true by accident, because the stock market regulators did not possess the information they needed to understand the stock markets. The unit of trading was now the microsecond, but the records kept by the exchanges were by the second.
[Brad] read the report once and then never looked at it again. “Once you get a sense of the speed with which things are happening, you realize that explanations like this–someone hitting a button–are not right,” he said. You want to see a single time-stamped sheet of every trade. To see what followed from what. Not only does it not exist, it can’t exist, as currently configured."
No one could say for sure what caused the flash crash–for the same reason no one could prove that high-frequency traders were front-running the orders of ordinary investors. The data didn’t exist. But Brad sensed that the investment community was not persuaded by the SEC’s explanation and by the assurances of the stock exchanges that all was well inside them. A lot of them asked the same question he was asking himself: Isn’t there a much deeper question of how this one snowball caused a deadly avalanche?
After the flash crash, Brad no longer bothered to call up investors to set up meetings. His phone rang off the hook. “What the flash crash did,” said Brad, “was it opened the buy side’s willingness to understand what was going on. Because their bosses started asking questions. Which meant that our telling the truth, and explaining it to them, fit perfectly.”
A few months later, in September 2010, another strange, albeit more obscure, market event occurred, this time in the Chicago suburbs. A sleepy stock exchange called the CBSX, which traded just a tiny fraction of total stock market volume, announced that it was going to invert the usual system of fees and kickbacks. It was now going to pay people to “take” liquidity and charge people to “make” it. Once again, this struck Brad as bizarre: Who would make markets on exchanges if they had to pay to do it? But then the CBSX exploded with activity. Over the next several weeks, for example, it handled a third of the total volume of the shares traded in Sirius, the satellite radio company. Brad knew that Sirius was a favorite stock of HFT firms–but he couldn’t understand why it was suddenly trading in huge volume in Chicago. Obviously, when they saw they could be paid to “take” on the CBSX, the big Wall Street brokers all responded by reprogramming their routers so that their customers’ orders were sent to the CBSX. But who was on the other side of their trades, paying more than ever had been paid for the privilege?
That’s when Ronan told Brad about a new company called Spread Networks. Spread Networks, as it turned out, had tried to hire Ronan to sell its precious line to high-frequency traders.
Ronan offered an explanation for what had just happened on the CBSX: Spread Networks had flipped its switch and turned itself on just two weeks earlier. CBSX then inverted its pricing. By inverting its pricing–by paying brokers to execute customers’ trades for which they would normally be charged a fee–the exchange enticed the brokers to send their customers' orders to the CBSX so that they might be front-run back to New Jersey by high-frequency traders using Spread Networks. The information that high-frequency traders gleaned from trading with investors in Chicago they could use back in the markets in New Jersey. It was no very much worth it to them to pay the CBSX to “make” liquidity. It was exactly the game they had played on BATS, of enticing brokers to reveal their customers' intentions so that they might exploit them elsewhere. But racing a customer order from Weehawken to other points in New Jersey was hard compared to racing it from Chicago on Spread’s new line.
most of the hundreds of big-time investors with whom Brad and Ronan spoke had the same reaction as T. Rowe Price’s Mike Gitlin: They knew something was very wrong, but they didn’t know what, and now that they knew they were outraged. “Brad was that honest broker,” said Gitlin. […] “Your biggest competitive advantage is that you don’t want to fuck me.”
every brokerage firm strongly encouraged investors who wanted to buy or sell big chunks of stock to do so in that firm’s dark pool. In theory, the brokers were meant to find the best price for their customers. If the customer wanted to buy shares in Chevron, and the best price happened to be on the New York Stock Exchange, the broker was not supposed to stick the customer with a worse price inside their dark pool. But the dark pools were opaque. Their rules were not published. No outsider could see what went on inside them. It was entirely possible that a broker’s own traders were trading against the customers in the dark pool: There were no rules against it. And while the brokers often protested that there were no conflicts of interest inside their dark pools, all the dark pools exhibited the same property: A huge percentage of the customer orders sent into a dark pool were executed inside the pool.
A broker was expected to find the best possible price in the market for his customer. The Goldman Sachs dark pool–to take one example–was less than 2 percent of the entire stock market. So why did nearly 50 percent of the customer orders routed into Goldman’s dark pool end up being executed inside that pool–rather than out in the wider market? Most of the brokers' dark pools constituted less than 1 percent of the entire market, and yet somehow those brokers found the best price for their customers between 15 and 60 percent of the time. (So-called rates of internalization varied from broker to broker.) And because the dark pool was not required to say exactly when it had executed a trade, and the broker did not typically tell his investors where it had executed a trade, much less the market conditions at the moment of execution, the customer lived in darkness.

Chapter 4: Tracking the Predator

[John Schwall started looking for] the cause of the problem Thor had solved: How was it legal for a handful of insiders to operate at faster speeds than the rest of the market and, in effect, steal from investors? He soon had his answer: Regulation National Market System. Passed by the SEC in 2005 but not implemented until 2007, Reg NMS, as it became known, required brokers to find the best market prices for the investors they represented.
Up till then the various brokers who handled investors' stock market orders had been held to the loose standard of “best execution.” What that meant in practice was subject to interpretation.
Reg NMS replaced the loose notion of best execution with the tight legal one of “best price.” To define best price, Reg NMS relied on the concept of the National Best Bid and Offer, known as the NBBO. If an investor wished to buy 10,000 shares of Microsoft, and 100 shares were offered on the BATS exchange at $30 a share, while the full 10,000 listed on the other twelve exchanges were offered at $30.01, his broker was required to purchase the 100 shares on BATS at $30 before moving on to the other exchanges.
The regulation also made it far easier for high-frequency traders to predict where brokers would send their customers' orders, as they must send them first to the exchange that offered the best market price.
The new law required a mechanism for taking the measure of the entire market–for creating the National Best Bid and Offer–by compiling all the bids and offers for all U.S. stocks in one place. That place, inside some computer, was called the Securities Information Processor.
Reg NMS was intended to create equality of opportunity in the U.S. stock market. Instead it institutionalized a more pernicious inequality. A small class of insiders with the resources to create speed were now allowed to preview the market and trade on what they had seen.
Thus–for example–the SIP might suggest to the ordinary investor in Apple Inc. that the stock was trading at 400-400.01. The investor would then give his broker his order to buy 1,000 shares at the market price, or $400.01. The infinitesimal period of time between the moment the order was submitted and the moment it was executed was gold to the traders with faster connections. How much gold depended on two variables: a) the gap in time between the public SIP and the private ones and b) how much Apple’s stock price bounced around. The bigger the gap in time, the greater the chance that Apple’s stock price would have moved; and the more likely that a fast trader could stick an investor with an old price. That’s why volatility was so valuable to high-frequency traders: It created new prices for fast traders to see first and to exploit. It wouldn’t matter if some people in the market had an early glimpse of Apple’s price if the price of Apple’s shares never moved.
Apple’s stock moved a lot, of course. In a paper published in February 2013, a team of researchers at the University of California, Berkeley, showed that the SIP price of Apple stock and the price seen by traders with faster channels of market information differed 55,000 times in a single day. That meant that there were 55,000 times a day a high-frequency trader could exploit the SIP-generated ignorance of the wider market. Fifty-five thousand times a day, he might buy Apple shares at an outdated price, then turn around and sell them at the new, higher price, exploiting the ignorance of the slower-footed investor on either end of his trades.
[John Schwall had actually helped Bank of America] build so-called smart order routers that could figure out which exchange had the official best price of any given stock (the NBBO) and send the customers' orders to that exchange. By complying with Reg NMS, he now understood, the smart order routers simply marched investors into various traps laid for them by high-frequency traders.
[From Schwall’s research he’d] learned several important things, he told his colleagues. First, there was nothing new about the behavior they were at war with: The U.S. financial markets had always been either corrupt or about to be corrupted. Second, there was zero chance that the problem would be solved by financial regulators; or, rather, the regulators might solve the narrow problem of front-running in the stock market by high-frequency traders, but whatever they did to solve the problem would create yet another opportunity for financial intermediaries to make money at the expense of investors.
[RBC’s] marketing department proposed to Brad, as a way to get some media attention for Thor, that he apply for a Wall Street Journal Technology Innovation Award. […] he thought that he might use the Wall Street Journal to tell the world just how corrupt the U.S. stock market had become. His bosses at RBC, when they got wind of his plans, […] worried about their relationships with other Wall Street banks and with the public exchanges.
“I had about eight things I wanted to say to the Journal,” said Brad. “By the time I got through all these meetings, there was nothing to say. I was only allowed to say one of them–that we had found a way to route orders so they arrived at the exchanges simultaneously.”
Before Brad said anything at all to the Wall Street Journal, RBC’s upper management felt they needed to inform the U.S. regulators of what little he planned to say. They asked Brad to prepare a report on Thor for the SEC and then flew themselves down from Canada to join him in a big meeting with the SEC’s Division of Trading and Markets staff. […] When he was finished, an SEC staffer said, What you are doing is not fair to high-frequency traders. You’re not letting them get out of the way.
After the meeting, RBC conducted a study, never released publicly, in which they found that more than two hundred SEC staffers since 2007 had left their government jobs to work for high-frequency trading firms or the firms that lobbied Washington on their behalf. Some of these people had played central roles in deciding how, or even whether, to regulate high-frequency trading. For instance, in June 2010, the associate director of the SEC’s Division of Trading and Markets, Elizabeth King, had quit the SEC to work for Getco. The SEC, like the public stock exchanges, had a kind of equity stake in the future revenues of high-frequency traders.
The argument in favor of high-frequency traders had beaten the argument against them to the U.S. regulators. It ran as follows: Natural investors in stocks, the people who supply capital to companies, can’t find each other. The buyers and sellers of any given stock don’t show up in the market at the same time, so they need an intermediary to bridge the gap, to buy from the seller and to sell to the buyer. The fully computerized market moved too fast for a human to intercede in it, and so the high-frequency traders had stepped in to do the job. Their importance can be inferred from their activity: In 2005 a quarter of all trades in the public stock markets were made by HFT firms; by 2008 that number has risen to 65 percent. Their new market dominance–so the argument went–was a sign of progress, not just necessary but good for investors. Back when human beings sat in the middle of the stock market, the spreads between the bids and the offers of any given stock were a sixteenth of a percentage point. Now that computers did the job, the spread, at least in the more actively traded stocks, was typically a penny, or one-hundredth of 1 percent. That, said the supporters of high-frequency trading, was evidence that more HFT meant more liquidity.
The arguments against the high-frequency traders hadn’t spread nearly so quickly–at any rate, Brad didn’t hear them from the SEC. A distinction cried out to be made, between “trading activity” and “liquidity.” A new trader could leap into a market and trade frantically inside it without adding anything of value to it. Imagine, for instance, that someone passed a rule, in the U.S. stock market as it is currently configured, that required every stock market trade to be front-run by a firm called Scalpers Inc. Under this rule, each time you went to buy 1,000 shares of Microsoft, Scalpers Inc. would be informed, whereupon it would set off to buy 1,000 shares of Microsoft offered in the market and, without taking the risk of owning the stock for even an instant, sell it to you at a higher price. Scalpers Inc. is prohibited from taking the slightest market risk; when it buys, it has the seller firmly in hand; when it sells, it has the buyer in hand; and at the end of every trading day, it will have no position at all in the stock market. Scalpers Inc. trades for the sole purpose of interfering with trading that would have happened without it. In buying from every seller and selling to every buyer, it winds up: a) doubling the trades in the marketplace and b) being exactly 50% of that booming volume. It adds nothing to the market but at the same time might be mistaken for the central player in that market.
This state of affairs, as it happens, resembles the United States stock market after the passage of Reg NMS. From 2006 to 2008, high-frequency traders' share of total U.S. stock market trading doubled, from 26 percent to 52 percent–and it has never fallen below 50 percent since then. The total number of trades made in the stock market also spiked dramatically, from roughly 10 million per day in 2006 to just over 20 million per day in 2009.
“Liquidity” was one of those words Wall Street people threw around when they wanted the conversation to end, and for brains to go dead, and for all questioning to cease. A lot of people used it as a synonym for “activity” or “volume of trading,” but it obviously needed to mean more than that, as activity could be manufactured in a market simply by adding more front-runners to it. To get a useful understanding of liquidity and the likely effects of high-frequency trading on it, one might better begin by studying the effect on investors' willingness to trade once they sense that they are being front-run by this new front-running entity. Brad himself had felt the effect: When the market as displayed on his screens became illusory, he became less willing to take risk in that market–to provide liquidity. He could only assume that every other risk-taking intermediary–every other useful market participant–must have felt exactly the same way.
The argument for HFT was that it provided liquidity, but what did this mean? “HFT firms go home flat every night,” said Brad. “They don’t take positions. They are bridging an amount of time between buyers and sellers that’s so small that no one even knows it exists.” After the market was computerized and decimalized, in 2000, spreads in the market had narrowed–that much was true. Part of that narrowing would have happened anyway, with the automation of the stock market, which made it easier to trade stocks priced in decimals rather than in fractions. Part of that narrowing was an illusion: What appeared to be the spread was not actually the spread. The minute you went to buy or sell at the stated market price, the price moved. What Scalpers Inc. did was to hide an entirely new sort of activity behind the mask of an old mental model–in which the guy who “makes markets” is necessarily taking market risk and providing “liquidity.” But Scalpers Inc. took no market risk."
In spirit Scalpers Inc. was less a market enabler than a weird sort of market burden. Financial intermediation is a tax on capital; it’s the toll paid by both the people who have it and the people who put it to productive use. Reduce the tax and the rest of the economy benefits. Technology should have led to a reduction in this tax; the ability of investors to find each other without the help of some human broker might have eliminated the tax altogether. Instead this new beast rose up in the middle of the market and the tax increased–by billions of dollars. Or had it? To measure the cost to the economy of Scalpers Inc., you needed to know how much money it made. That was not possible. The new intermediaries were too good at keeping their profits secret. Secrecy might have been the signature trait of the entities who now sat at the middle of the stock market. You had to guess what they were making from what they spent to make it.
Apart form taking some large sum of money out of the market, and without taking risk or adding anything of use to that market, Scalpers Inc. had other, less intended consequences. Scalpers Inc. inserted itself into the middle of the stock market not just as an unnecessary middleman but as a middleman with incentives to introduce dysfunction into the stock market. Scalpers Inc. was incentivized, for instance, to make the market as volatile as possible. The value of its ability to buy Microsoft from you at $30 a share and to hold the shares for a few microseconds–knowing that, even if the Microsoft share price began to fall, it could turn around and sell the shares at $30.01–was determined by how likely it was that Microsoft’s share price, in those magical microseconds, would rise in price. The more volatile Microsoft’s share price, the higher Microsoft’s stock price might move during those microseconds, and the more Scalpers Inc. would be able to scalp. One might argue that intermediaries have always profited from market volatility, but that is not really true. The old specialists on the New York Stock Exchange, for instance, because they were somewhat obliged to buy in a falling market and to sell in a rising one, often found that their worst days were the most volatile days. They thrived in times of relative stability.
Another incentive of Scalpers Inc. is to fragment the marketplace: The more sites at which the same stocks changed hands, the more opportunities to front-run investors from one site to another. The bosses at Scalpers Inc. would thus encourage new exchanges to open, and would also encourage them to place themselves at some distance form each other. Scalpers Inc. also had a very clear desire to maximize the difference between the speed of their private view of the market and the view afforded the wider public market. The more time that Scalpers Inc. could sit with some investor’s stock market order, the greater the chance that the price might move in the interim. Thus an earnest employee of Scalpers Inc. would look for ways either to slow down the public’s information or to speed up its own.
The final new incentive introduced by Scalpers Inc. was perhaps the most bizarre. The easiest way for Scalpers Inc. to extract the information it needed to front-run other investors was to trade with them. At times it was possible to extract the necessary information without having to commit to a trade. That’s what the “flash order” scandal had been about: high-frequency traders being allowed by the exchanges to see other people’s orders before anyone else, without any obligation to trade against them. But for the most part, if you wanted to find out what some big investor was about to do, you needed to do a little bit of it with him. For instance, to find out that, say, T. Rowe Price wanted to buy 5 million shares of Google Inc., you needed to sell some Google to T. Rowe Price. That initial market contract between any investor and Scalpers Inc. was like the bait in a trap–a loss leader. For Scalpers Inc., the goal was to spend as little as possible to acquire the necessary information–to make those initial trades, the bait, as small as possible.
The financial crisis brought with it a great deal of stock market volatility; perhaps people just assumed that there was supposed to be an unusual amount of drama in the stock market evermore.
The Royal Bank of Canada had tested the effects on stock market volatility of using Thor, which stymied front-runners, rather than the standard order routers used by Wall Street, which did not. The sequential cost-effective router responded to the kickbacks and fees of the various exchanges and went to those exchanges first that paid them the most to do so. The spray router–which, as its name suggests, just sprayed the market and took whatever stock was available, or tried to–did not make any effort to compel a stock market order to arrive at the different exchanges simultaneously. Every router, when it bought stock, tended to drive the priced of that stock a bit higher. But when the stock had settled–say, ten seconds later–it settled differently with each router. The sequential cost-effective router caused the share price to remain higher than the spray router did, and the spray router caused it to move higher than Thor did.
The new choppiness in the public U.S. stock markets was spreading to other financial markets, as they, too, embraced high-frequency traders. It was what investors most noticed: They were less and less able to buy and sell big chunks of stock in a gulp. Their frustration with the public stock exchanges had led the big Wall Street banks to create private exchanges: dark pools.
[Rich Gates, who ran the TFS Capital mutual fund] and his colleagues devised a test to see if there was anything in this new stock market [of algorithms and talk of fast trading] to fear. The test, specifically, would show him if, when he entered an order into one of Wall Street’s dark pools, he wound up getting ripped off by some unseen predator. He started by identifying stocks that didn’t trade very often. Chipotle Mexican Grill, for instance. He sent in an order to a single Wall Street dark pool to buy that stock at the “mid-market” price. Say, for example, that the shares of Chipotle Mexican Grill were trading at 100-100.10. Gates would submit his bid to buy a thousand shares of Chipotle at $100.05. There it would normally just sit until some other investor came along and lowered his price from $100.10 to $100.05. Gates didn’t wait for that to happen. Instead, a few seconds later, he sent a second order to one of the public exchanges, to sell Chipotle at $100.01.
What should have happened next was that his order in the dark pool should have been filled at $100.01, the official new best price in the market. He should have been able to buy from himself the shares he was selling at $100.01. But that’s not what happened. Instead, before he could blink his eye, he had made two trades. He had bought Chipotle from someone inside the Wall Street dark pool at $100.05 and sold it to someone else on the public exchange for $100.01. He’d lost 4 cents by, in effect, trading with himself. Only he hadn’t traded with himself; some third party obviously used the sell order he had sent to the public exchange to exploit the buy order he had sent to the dark pool.
Gates and his colleagues wound up making hundreds of such tests, with their own money, in several Wall Street dark pools. In the first half of 2010 there was only one Wall Street firm in whose dark pool the test came back positive: Goldman Sachs. In the Goldman dark pool, Sigma X, he got ripped off a bit more than half the time e ran the test. As Gates traded in lightly traded stocks, and high-frequency trading firms were overwhelmingly interested in heavily traded ones, these tests would have been vastly more likely to generate false negatives than false positives.
[Gates published about this in the Wall Street Journal, which] led a person close to both the BATS exchange and Credit Suisse to get in touch with Gates with a suggestion: Run your tests again, specifically on the BATS exchange and the Credit Suisse dark pool called Crossfinder. Just to see. Toward the end of 2010, Gates ran another round of tests. […] “When we did it the first time,” he said, “it worked at Goldman but nowhere else. When we did it six months later it didn’t work at Goldman, but it worked everywhere else.”
[When Schwall and Brad found out that banks were selling access to dark pools, they asked themselves the question] Why would anyone pay for access to the customers' orders inside a Wall Street bank’s dark pool? The straight answer was that a customer’s stock market order, inside a dark pool, was fat and juicy prey. The order was typically large, and its movements were especially predictable: Each Wall Street bank had its own detectable pattern for handling orders. The order was also slow, because of the time it was forced to spend inside the dark pool before accessing the wider market. As Brad had put it, “You could front-run an order in a dark pool on a bicycle.” The pension fund trying to buy 100,000 shares of Microsoft could, of course, specify that the Wall Street bank not take its orders to the public exchanges at all but simply rest it, hidden inside the dark pool. But an order hidden inside a dark pool wasn’t very well hidden. Any decent high-frequency trader who had paid for a special connection to the pool would ping the pool with tiny buy and sell orders in every listed stock, searching for activity. Once they’d discovered the buyer of Microsoft, they’d simply wait for the moment when Microsoft ticked lower on the public exchanges and sell it to the pension fund in the dark pool at the stale, higher “best” price (as Rich Gates’s tests had demonstrated). It was riskless, larcenous, and legal–made so by Reg NMS. The way Brad had described it, it was as if only one gambler were permitted to know the scores of last week’s NFL games, with no one else aware of his knowledge. He places bets in the casino on every game and waits for other gamblers to take the other side of those bets. There’s no guarantee that anyone will do so; but if they do, he’s certain to win.
[After reverse engineering the organization of the dark pools at Credit Suisse], the bank that went to the most trouble to sell itself as safe to investors, Brad decided that the game was probably all over inside all the big Wall Street banks. All of them, one way or another, were probably using the unequal speeds in the market to claim their share of the prey. He further assumed that the big Wall Street banks must have stumbled upon his solution to high-frequency front-running, and must have chosen not to use it, because they had too great a stake in the profits generated by that front-running. “It became very obvious to me why we were the first to discover Thor, because we weren’t,” he said. “What that meant to me was that the problem was going to be much, much harder to solve. It also told me why the clients were so in the dark, because the clients rely on brokers for information.” Creating an exchange designed to protect the pretty from the predator would mean starting a war on Wall Street–between the banks and the investors they claimed to represent.

Chapter 5: Putting a Face on HFT

What [Sergey Aleynikov] did know about Goldman’s business was that the firm’s position in the world of high-frequency trading was insecure. “The traders were always afraid of the small HFT shops,” as he put it. He was making Goldman’s bulky, inefficient system faster, but he could never make it as fast as a system built from scratch, without the burden of 60 million lines of old code underneath it. Or a system that, to change it in any major way, did not require six meetings and signed documents from informational security officers. Goldman hunted in the same jungle as the small HFT firms, but it could never be as quick or as nimble as those firms: NO big Wall Street bank could. The only advantage a big bank enjoyed was its special relationship to the prey: its customers. (As the head of one high-frequency trading firm put it, “When one of these people from the banks interviews with us for a job, he always talks about how smart his algos are, but sooner or later he’ll tell you that without his customer he can’t make any money.")
After a few months working on the forty-second floor at One New York Plaza, Serge came to the conclusion that the best thing they could do with Goldman’s high-frequency trading platform was to scrap it and build a new one from scratch. His bosses weren’t interested. “The business model of Goldman Sachs was, if there is an opportunity to make money right away, let’s do that,” he says. “But if there was something long-term, they weren’t that interested. Something would change in the stock market–an exchange would introduce a new, complicated rule, for instance–and that change would create an immediate opportunity to make money. “They’d want to do it immediately,” says Serge. “But if you think about it, it’s just patching the existing system constantly.The existing code base becomes an elephant that’s difficult to maintain.”
That is how he spent the vast majority of his two years at Goldman, patching the elephant.
The funny thing was that Serge actually liked Adam Schlesinger, and most of the other people he worked with at Goldman. He liked less the environment the firm created for them to work in. “Everyone lived for the year-end number,” he said. “You get satisfied when the bonus is sizable and you get not satisfied with the number is not. Everything there is very possessive.” It made no sense to him the way people were paid individually for achievements that were essentially collective achievements.

Chapter 6: How to Take Billions from Wall Street

What had gone wrong, in [Don Bollerman]’s view, wasn’t all that surprising or complicated.It had to do with human nature, and the power of incentives. The rise of high-frequency trading–and its ability to gain an edge on the rest of the market–had created an opportunity for new exchanges, like BATS and Direct Edge. By giving HFT what it wanted (speed, in relation to the rest of the market; complexity only HFT understood; and payment to brokers for their customers' orders, so that HFT had something to trade against), the new stock exchanges had stolen market share from the old stock exchanges. Don couldn’t speak for NYSE, but he had watched Nasdaq respond by giving HFT firms what they asked for–and then figuring out how to charge them for it. “It was almost like you couldn’t do anything about it,” he said. “We did all this speed, and I don’t think we fully understood what it was being used for. We just thought, The new rules caused people to have a new experience and then new wants and needs.” Nasdaq had become a public company in 2005, a year after Don had joined it. It had earnings targets to hit; it was incentivized to make decisions, and to make changes in the nature of the exchange, with a focus on their short-term consequences. “It’s hard to be forward-thinking when the whole of corporate America is about the next quarter’s earnings,” said Don. “It went from ‘Is this good for the market?’ to ‘Is this bad for the market?’ And then it slides to: ‘Can we get this through the SEC?’ The demon in this part of the story is expediency.” By late 2011, when Bollerman quit his job (“I felt there was a lack of leadership”), more than two-thirds of Nasdaq’s revenues derived, one way or another, from high-frequency trading platforms.
To Don’s way of thinking, you were never going to change human nature–though you might alter the environment in which it expressed itself.
Brad wasn’t in the market for self-righteousness, or for people who defined themselves by their fine moral sentiment. “Disillusion isn’t a useful emotion,” he said. “I need soldiers.” Don was a soldier.
Their new exchange needed a name. They called it the Investors Exchange, which wound up being shortened to IEX.
- In the interest of clarity, they’d hoped to preserve the full name, but they discovered a problem doing so when they set out to create an Internet address: investorsexchange.com. To avoid that confusion, they created another.
[Constantine Sokoloff] had a theory about why so many Russians had wound up inside high-frequency trading. The old Soviet educational system channeled people away from the humanities and into math and science. The old Soviet culture also left its former citizens oddly prepared for Wall Street in the early twenty-first century. The Soviet-controlled economy was horrible and complicated but riddled with loopholes. Everything was scarce; everything was also gettable, if you knew how to get it. “We had this system for seventy years,” said Constantine. “People learn to work around the system. The more you cultivate a class of people who know how to work around the system, the more people you will have who know how to do it well. All of the Soviet Union for seventy years were people who are skilled at working around the system.” The population was thus well suited to exploit megatrends in both computers and the United States financial markets. After the fall of the Berlin Wall, a lot of Russians fled to the United States without a lot of English; one way to make a living without having to converse with the locals was to program their computers.
Aspects of the existing stock exchanges obviously incentivized bad behavior.
Rebates, for instance: The maker-taker system of fees and kickbacks used by all of the exchanges was simply a method for paying the big Wall Street banks to screw the investors whose interests they were meant to guard. The rebates were the bait in the high-frequency traders' flash traps. The moving parts of the traps were order types.
- The market order is the first and simplest type. Say, for instance, an investor wishes to buy 100 shares of Procter & Gamble. When he submits his order, the market for the shares in P&G is, say, 80-80.02. If he submits a market order, he will pay the offering price–in this case, $80.02 per share. But a market order comes with a risk: that the market will move between the time the order is submitted and the time it reaches the market. The flash crash was a dramatic illustration of that risk: Investors who submitted market orders wound up paying $100,000 a share for P&G and selling those same shares for a penny apiece. To control the risk of a market order, a second order type was invented, the limit order. The buyer of P&G shares might say, for instance: “I’ll buy a hundred shares, with a limit of eighty dollars and three cents a share.” By doing so, he will ensure that he does not pay $100,000 a share; but this may lead to a missed opportunity–hey may not buy the shares at all, because he never gets the price he wanted. Another simple, and long-used, order type is “good ‘til canceled.” The investor who says he wants to buy 100 shares of P&G at $80 a share, “good ‘til canceled,” will never have to think about it again until he buys them, or does not.
All of the exchanged offered something called a Post-Only order. A Post-Only order to buy 100 shares of Procter & Gamble at $80 a share says, “I want to buy a hundred shares of Procter & Gamble at eighty dollars a share, but only if I am on the passive side of the trade, where I can collect a rebate from the exchange.
A Hide Not Slide order–it was just one of maybe fifty such problems the Puzzle Masters [math nerds at IEX] solved–worked as follows: The trader said he was willing to buy the shares at a price ($80.03) above the current offering price ($80.02), but only if he was on the passive side of the trade, where he would be paid a rebate. He did this not because he wanted to buy the shares. He did this in case an actual buyer of stock–a real investor, channeling capital to productive enterprise–came along and bought all the shares offered at $80.02. The high-frequency trader’s Hide Not Slide order then established him as first in line to purchase P&G shares if a subsequent investor came into the market to sell those shares. This was the case even if the investor who had bought the shares at $80.02 expressed further demand for them at the higher price. A Hide Not Slide order was a way for a high-frequency trader to cut in line, ahead of the people who’d created the line in the first place, and take the kickbacks paid to whoever happened to be at the front of the line.
As they worked through the order types, they created a taxonomy of predatory behavior in the stock market. Broadly speaking, it appeared as if there were three activities that led to a vast amount of grotesquely unfair trading. The first they called “electronic front-running”–seeing an investor trying to do something in one place and racing him to the next.
The second they called “rebate arbitrage”–using the new complexity to game the seizing of whatever kickbacks the exchange offered without actually providing the liquidity that the kickback was presumably meant to entice. The third, and probably by far the most widespread, they called “slow market arbitrage.” This occurred when a high-frequency trader was able to see the price of a stock on one exchange, and pick off orders sitting on other exchanges, before the exchanges were able to react.
It wasn’t necessary to eliminate high-frequency traders; all that was needed was to eliminate the unfair advantages they had, gained by speed and complexity.
The obvious starting point was to prohibit high-frequency traders from doing what they had done on all the other exchanges–co-locating inside them, and getting the information about whatever happened on those exchanges before everyone else.
The idea was to establish the IEX computer that matched buyers and sellers (the matching engine) at some meaningful distance from the place traders connected to IEX (called the “point of presence”), and to require anyone who wanted to trade to connect to the exchange at that point of presence. If you placed every participant in the market far enough away from the exchange, you could eliminate most, and maybe all, of the advantages created by speed.
The only question was: Where to put the point of presence?
The delay needed only to be long enough for IEX, once it had executed some part of a customer’s buy order, to beat HFT in a race to any other shares available in the marketplace at the same price–that is, to prevent electronic front-running. It needed to be long enough, also, for IEX, each time a share price moved on any exchange, to process the change, and to move the prices of any orders resting on it, so that they didn’t get picked off.
The necessary delay turned out to be 320 microseconds; that was the time it took them, in the worst case, to send a signal to the exchange farthest from them, the NYSE in Mahwah. Just to be sure, they rounded it up to 350 microseconds.
The new stock exchange also cut off the food source for all identifiable predators.
The fiber routes through New Jersey that Ronan handpicked were chosen so that an order sent from IEX to the other exchanges arrived at them all at precisely the same time. (He thus achieved with hardware what Thor had achieved with software.
To “see” the prices on the other stock exchanges, IEX didn’t use the SIP or some phony improvement on the SIP but instead created their own private, HFT-like pictures of the entire stock market.
The 350-microsecond delay worked like a head start in a footrace. It ensured that IEX would be faster to see and react to the wider market than even the fastest high-frequency trader, thus preventing investors’ orders from being abused by changes in that market. In the bargain, it prevented high-frequency traders–who would inevitably try to put their computers nearer than everyone else’s to IEX’s in Weehawken–from submitting their orders onto IEX more quickly than everyone else.
To create this 350-microsecond delay, they needed to keep the new exchange roughly thirty-eight miles from the place the brokers were allowed to connect to the exchange.
Instead of running straight fiber between the two places, [they would] coil thirty-eight miles of fiber and stick it in a compartment the size of a shoebox to simulate the effects of the distance.
Creating fairness was remarkably simple. They would not sell to any one trader or investor the right to put his computers next to the exchange, or special access to data from the exchange. They would pay no kickbacks to brokers or banks that sent orders; instead, they’d charge both sides of any trade the same amount: nine one-hundredths of a cent per share (known as 9 “mils”). They’d allow just three order types: market, limit, and Mid-Point Peg, which meant that the investor’s order rested in between the current bit and offer of any stock.
Finally, to ensure that their own incentives remained as closely aligned as they could be with those of stock market investors, the new exchange did not allow anyone who could trade directly on it to own any piece of it: Its owners were all ordinary investors who needed first to hand their orders to brokers.
The Wall Street banks controlled not only the orders, and the informational value of those orders, but dark pools in which those orders might be executed. The banks took different approaches to milking the value of their customers’ orders. All of them tended to send the orders first to their own dark pools before routing them out to the wider market. Inside the dark pool, the bank could trade against the orders themselves; or they could sell special access to the dark pool to high-frequency traders. Either way, the value of the customers' orders was monetized.
If the Puzzle Masters were right, and the design of IEX eliminated the advantage of speed, IEX would reduce the value of investors' stock market orders to zero.

Chapter 7: An Army of One

[Zoran Perkov] also seemed to assume that his new colleagues would fail to understand the difference between what he could control and what he couldn’t. In one thirty-day span after he joined IEX, he shot out fifteen emails on this one subject–to hammer home the mystery inherent in any stock market technological failure. He even invited a speaker to come in to reinforce the point. “It was one of the few times that the people in the room wound up at each other’s throats,” said Braid. “The tech people were all agreeing with him, and the business people were saying, ‘If something melts down, how could it not be someone’s fault?'”
Initially Brad was mystified: How could a guy who thrived under pressure also have such a fear of being blamed if things went wrong? […] Brad realized something: “It comes from a sense of insecurity that comes from the fact that he will be more recognized when things go wrong than when things go right.” Brad further realized that the problem was not peculiar to Zoran but general to Wall Street technologists. The markets were now run by technology, but the technologists were still treated like tools. Nobody bothered to explain the business to them, but they were forced to adapt to its demands and exposed to its failures–which was, perhaps, why there had been so many more conspicuous failures.
Nasdaq’s famously talented engineers were an extreme Wall Street case. The constant pressure on Nasdaq’s tech guys to adapt the stock markets’ code to the needs of high-frequency traders had created a miserable, politicized workplace. The Nasdaq business guys foisted all these unreasonable demands on the tech guys and then, when the demands busted the system, blamed the tech guys for the failure. The tech guys all wound up with this abused animal quality to them. “You just have to unabuse them,” Brad explained, “and let them know they aren’t going to be blamed just because something goes wrong.” We all know that things will go wrong and it isn’t necessarily anyone’s fault.
Brad also heard what the big Wall Street banks were already saying to investors to dissuade them from sending orders to IEX: It’s too slow. For years, the banks had been selling the speed and aggression of their trading algos, along with the idea that, for an investor, slower always meant worse. They seemed to have persuaded themselves that the new speed of the markets actually helped their clients. They’d even dreamed up a technical-sounding name for an absence of speed: “duration risk.”
In 98.22 percent of all milliseconds, nothing at all happened in the U.S. stock market. To a computer, the market in even the world’s most actively traded stock was an uneventful, almost sleepy place. “Yes, your eyeballs think the markets are going fast,” Brad said. “They aren’t really going that fast.” The likelihood an investor would miss out on something important in a third of a millisecond was close to zero, even in the world’s most actively traded stock. “I knew it was bullshit to worry about milliseconds,” said Brad, “because if milliseconds were relevant, every investor would be in New Jersey.”
They were unlikely to miss any action as the result of a delay of one-third of a millisecond. They were the reason for all the action! “Every time a trade happens at the exchange, it creates a signal,” said Brad. “In the fifty milliseconds running up to it–total silence. Then there is an event. Then there is this massive reaction. Then a reaction to that reaction. The HFT algos on the other side are predicting what you’ll do next based on what you just did.” The activity peaked roughly 350 microseconds after an investor’s order triggered the feeding frenzy, or the time it took for HFT to send its orders from the stock exchange on which the investor had touched down to all of the others.
The arrival of the prey awakened the predator, who deployed his strategies–rebate arbitrage, latency arbitrage, slow market arbitrage.
These new pictures [from Josh Blackburn] showed [Brad] how the big Wall Street banks typically handled investors' stock market orders. Here’s how it worked: Say you are a big investor–a mutual fund or a pension fund–and you have decided to make a big investment in Procter & Gamble. You are acting on behalf of a lot of ordinary Americans who have given you their savings to manage. You call some broker–Bank of America, say–and tell them you’d like to buy 100,000 shares of Procter & Gamble. P&G’s shares are trading at, say, 82.95-82.97, with 1,000 shares listed on each side. You tell the big Wall Street bank you were willing to pay up to, say $82.97 a share. From that point on, you basically have no clue how your order–and the information it contains–is treated. Now Brad saw: The first thing the broker did was to ping IEX with an order to buy 100 shares, to see if IEX had a seller. This made total sense: You didn’t want to reveal you had a big buyer until you found a seller. What made a lot less sense was what many of the brokers did after they discovered the seller. They avoided him.
Say, for example, that IEX actually had a seller waiting on it–a seller of 100,000 shares at $82.96. Instead of coming in and trying to buy a much bugger chunk of P&G, the big bank just kept pinging IEX with tiny 100-share orders–or the bank vanished entirely. If the bank had simply sent IEX and order to buy 100,000 shares of P&G at $82.97, the investor would have purchased all the shares he wanted without driving up the price. Instead the bank had pinged away and–by revealing its insistent, noisy demand–goosed up the price of P&G’s stock, at the expense of the investor whose interests the bank was meant to represent. Adding to the injury, the bank typically wound up with only a fraction of the stock its customer wanted to buy.
“I thought, Why the hell would anyone do this? All you do is increase the chances that an HFT will pick up your signal.”
It was as if they wished to appear to be interacting with the entire stock market, while actually they were trying to prevent any trades from happening outside their own dark pools.
Brad now explained to the investors, who were of course paying the price for this behavior, the reasons that the banks behaved as they did. The most obvious was to maximized the chance of executing the stock market orders given to them by investors in their own dark pools. The less honestly a bank looked for P&G stock outside of its own dark pool, the less likely it was to find it. This evasiveness explained the banks' incredible ability to find, eventually, the other side of any trade inside their own dark pools.
The big Wall Street banks wanted to trade in their own dark pools not only because they made more money–on top of their commissions–by selling the right to HFT to exploit the orders inside their dark pools. They wanted to trade their orders inside their dark pools to boost the volumes in those pools, for appearances' sake.
A stock market was judged by the volume of trading that occurred on it,,nature of that volume. It was widely believed, for example that the bigger the average trade size on an exchange, the better the market was for an investor.
For example, to show that they were capable of hosting big trades, the exchanges published the number of “block” trades of more than 10,000 shares they facilitated. The New York Stock Exchange sent IEX a record of 26 small trades it had made after IEX had routed an order to it–and then published the result on the ticker tape as a single 15,000-share block. The dark pools were even worse, as no one but the banks that ran them had a clear view of what happened inside them. The banks all published their own self-generated stats on their own dark pools: Every bank ranked itself #1.
The banks did not merely manipulate the relevant statistics in their own dark pools; they often to undermine the stats of their competitors. That was another reason the banks were sending IEX orders in tiny 100-share lots: to lower the average trade size in a market that competed with the banks' dark pools. A lower average trade size made IEX’s stats look bad–as if IEX were heavily populated by high-frequency traders.
Wondering if the broker was spreading news of his buy order elsewhere, Brad turned his attention to the consolidated tape of all trades that occurred in the U.S. stock market.
For each trade on IEX, he’d spotted a nearly identical trade that had occurred at nearly the same time in some other market.
He’d see a trade on IEX for 131 shares, of, say, Procter & Gamble, and then he’d see in some other market, exactly the same trade–131 shares of Procter & Gamble–within a few milliseconds, but at a slightly different price. It happened over and over again. He also noticed that, in each case, on one side of the trade was a broker who had rented out his pipes to a high-frequency trader.
Up till that point, most of the predation they had uncovered occurred when stock prices moved. A stock went up or down; the high-frequency guys found out before everyone else and took advantage of them. Roughly two-thirds of all stock market trades took place without moving the price of the stock–the trade happened at the seller’s offering price, or the buyer’s bidding price, or in between; afterwards, the bid and offering price remained the same as they had been before. What Brad now saw was how HFT, with the help of the banks, might exploit investors even when the stock price was stable. Say the market for Procter & Gamble’s shares was 80.50-80.52, and the quote was stable–the price wasn’t about to change. The National Best Bid was $80.50, and the National Best Offer was $80.52, and the stock was just sitting there. A seller of 10,000 Procter & Gamble shares appeared on IEX. IEX tried to price the orders that rested on it at the midpoint (the fair price), and so the 10,000 shares were being offered at $80.51. Some high-frequency trader would come into IEX–it was always a high-frequency trader–and chip away at the order: 131 shares here, 189 shares there. But elsewhere in the market, the same HFT was selling the shares–131 shares here, 189 shares there–at $80.52. On the surface, HFT was performing a useful function, building a bridge between buyer and seller. But the bridge itself was absurd. Why didn’t the broker who controlled the buy order simply come to IEX on behalf of his customer and buy, more cheaply, the shares offered?
The Wall Street banks were failing to send their customers' orders to the rest of the marketplace. An investor had given a Wall Street bank an order, say, to buy 10,000 shares of P&G. The bank had sent it to its dark pool with instructions for the order to stay there, aggressively priced, at $80.52. The bank was boosting its dark pool stats–and also charging some HFT a fee rather than paying a fee to another exchange–but it was also ignoring whatever else was happening in the market. In a functional market, the investors would simply have met in the middle and traded with each other at a price of $80.51. The price of the stock needn’t have moved a penny. The unnecessary price movement–caused by the screwed-up stock market–also played into HFT’s hands. Because high-frequency traders were always the first to detect any stock price movements, they were able to exploit, with other strategies, investors' ignorance of the fact that the market price had changed. The original false note struck by the big Wall Street bank–the act of avoiding making trades outside its own dark pool–became the prelude to a symphony of scalping. “We’re calling this ‘dark pool arbitrage,'” said Brad.
It was astonishing, when you stopped to think about it, how aggressively capitalism protected its financial middleman, even when he was totally unnecessary. Almost magically, the banks had generated the need for financial intermediation–to compensate for their own unwillingness to do the job honestly.
It wasn’t easy being Brad Katsuyama–to try to effect some practical change without a great deal of fuss, w hen the change in question was, when you got right down to it, a radical overhaul of a social order. Brad was not by nature a radical. He was simply in possession of radical truths.

Chapter 8: The Spider and the Fly

Epilogue: Riding the Wall Street Trail

There was a connection between Serge Aleynikov and Goldman’s behavior on December 19, 2013.
High-frequency trading had a winner-take-all aspect: The fastest predator took home the fattest prey. By 2013 the people charged with determining Goldman’s stock market strategy had concluded that Goldman wasn’t very good at this new game, and that Goldman was unlikely ever to be very good at it. The high-frequency traders would always be faster than Goldman Sachs–or any other big Wall Street bank.
The trouble for any big Wall Street bank wasn’t simply that a big bureaucracy was ill-suited to keeping pace with rapid technological change, but that the usual competitive advantages of a big Wall Street bank were of little use in high-frequency trading. A big Wall Street bank’s biggest advantage was its access to vast amounts of cheap risk capital and, with that, its ability to survive the ups and downs of a risky business. That meant little when the business wasn’t risky and didn’t require much capital.
A big Wall Street bank really had only one advantage in an ever-faster financial market: first shot at its own customers’ stock market trades. So long as the customers remained inside the dark pool, and in the dark, the bank might profit at their expense. But even here the bank would never do the job as efficiently or thoroughly as a really good HFT.
The new structure of the U.S. stock market had removed the big Wall Street banks from their historic, lucrative role as intermediary. At the same time it created, for any big bank, some unpleasant risks: that the customer would somehow figure out what was happening to his stock market orders. And that the technology might somehow go wrong. If the markets collapsed, or if another flash crash occurred, the high-frequency traders would not take 85 percent of the blame, or bear 85 percent of the costs of the inevitable lawsuits. The banks would bear the lion’s share of the blame and the costs. The relationship of the big Wall Street banks to the high-frequency traders, when you thought about it, was a bit like the relationship of the entire society to the big Wall Street banks. When things went well, the HFT guys took most of the gains; when things went badly, the HFT guys vanished and the banks took the losses.