Index Funds: Their Role & Impact on Oil and Other Commodity Prices

AgMRC Renewable Energy Newsletter
August 2008

Dr. Robert WisnerRobert Wisner                                                                  
Professor Emeritus BioFuels Economist
Ag Marketing Resources Center
Iowa State University


A recent development that is believed by some to have significant impact on energy prices is the movement of large “inflation hedge” funds into energy and other commodity futures markets for long-term investments.  These funds typically purchase a portfolio of various commodity futures contracts including metals, energy, grain, livestock, and oilseeds.  Their portfolios are designed to approximately match the composition of major published commodity price indices such as the Goldman Sachs Commodity Index that is traded on the Chicago Mercantile Exchange (CME) or the CRB Index Futures that is traded on the New York Board of Trade (NYBOT).  They are a commodity equivalent of stock index funds such as the Russell 2000 or the S&P 500, both of which are publicly traded.  Monetary sources for the large inflow of funds into commodity futures include pension funds and other large U.S. and foreign investors such as insurance companies.

Although there are similarities between commodity and stock index funds, there also are very important differences.  For example, stock funds trade the value of a company while commodity funds trade the price of the raw material for industrial processing – such as copper, crude oil, corn, or soybeans.  A surge of excess money into the stock market affects only the value of the company or companies involved.  A surge of excess money into commodity futures can affect the price of commodities which will affect the profitability of entire industries, at least for a period of time, with higher costs being passed on to consumers.

Commodity Futures Market Functions

Commodity futures markets perform three major functions: (1) price or value discovery, (2) allocation of resources into or out of commodity production and utilization, and (3) risk-management for producers and users of their respective commodity.

Price Discovery
Price discovery occurs as buyers and sellers from around the world trade in the commodity futures market, basing their decisions on current and future perceptions of supply and demand conditions.  Those perceptions can change from day to day as new information becomes available.  The markets continually search for an equilibrium, where the quantity demanded at a specific price just matches the quantity suppliers are willing to sell.

Resource Allocation
Futures markets can and often do over-react on either the upside or the downside.  When that happens, if prices are above the long-run equilibrium, users will reduce their consumption of the commodity and producers will take steps to increase the amount of resources being used for production – such as land, fertilizer and other inputs in the example of crop production.   For crude oil, more resources may flow into exploration for oil, and into increased investment in drilling equipment.  Biofuels are an excellent example of how high prices have stimulated the flow of resources into expanded production.  Time lags usually are involved for both the supply and the demand side of this process.

When markets work properly, and prices truly reflect supply/demand conditions, these prices send an accurate and important signal for allocating resources.  However, when markets do not work properly and prices are distorted, the distorted prices send an inaccurate signal that misallocates resources.  The harm to the economy as well as producers and users of the commodity from the misallocation of resources can be substantial.

Risk Management
Risk-management has been the main justification for existence of futures markets.  Otherwise, many would view them as another form of commercialized gambling.   Hedging is the risk-management mechanism that links futures prices to the underlying physical commodity market.  Hedgers traditionally have been producers or users of the commodity who need to protect profit margins in their business by protecting or locking-in prices for their product or input.  They do this by using the futures market as a temporary substitute for a later physical commodity market transaction.

The CFTC definition of hedging is as follows:

The Commission and exchanges grant exemptions to their position limits for bona fide hedging, as defined in CFTC Regulation 1.3(z), 17 CFR 1.3(z). A hedge is a derivative transaction or position that represents a substitute for transactions or positions to be taken at a later time in a physical marketing channel.

Hedges must reduce risk for a commercial enterprise and must arise from a change in the value of the hedger's (current or anticipated) assets or liabilities. For example, a short hedge includes sales for future delivery (short futures positions) that do not exceed its physical exposure in the commodity in terms of inventory, fixed-price purchases and anticipated production over the next 12 months.

Consider the case of a grain elevator buying grain from farmers and planning to store it for several months.  A traditional elevator may have operated on a gross margin of 10 to 15 cents per bushel, which had to cover operating costs and its profit margin.  In grain markets, prices can change by much more than that in a very short time, thus completely eliminating the elevator’s profit and creating a substantial loss.  In the terminology of the futures market, the elevator is long the physical commodity when it buys grain.  It has grain that will need to be sold later.  To hedge, it sells futures contracts (contracts for future delivery).  Later, when it sells grain in the physical market, it buys back the futures contract(s) to close out its hedge.  Thus the hedge involves a long cash grain or physical commodity position and a short (or sold) futures position.  With changing prices, these opposite positions tend to offset each other leaving the hedger in the same relative position as when the hedge was implemented.  The Chicago Board of Trade originally was developed for this purpose.  A type of futures market developed much earlier for rice in Japan, also for the same purpose.

Traditional Futures Trading

First we must understand that futures prices affect the actual commodity prices.  Delivery on futures contracts is not required and most contracts are not held for physical delivery.  But enough typically are held for delivery so that the physical commodity price at the delivery points matches the futures contract price during its delivery period.  This process forces cash (physical commodity) and futures prices to converge if the market is functioning properly.  Thus, movements in futures prices affect physical commodity prices.

Futures markets typically have had two types of traders: hedgers and speculators.  Hedgers, as described above, are those who use the market for risk protection.  Speculators are essential to the market because they give it liquidity, allowing hedgers to move in and out of the futures market easily at any time.  Traditional speculators trade on either side of the market, buying or selling futures contracts.  Their purpose for being in the market is to profit from price changes, which can be either short-term or long-term.

Speculators have limits on the number of futures contracts they can hold at any given time.  These limits have been increased substantially in the last 15 years as trading volumes have increased.  The size-of-position limits for speculators are to keep any one or small group of traders from obtaining a large non-hedging position that will distort prices.  There have been cases in the past where the Commodity Futures Trading Commission (CFTC), the government regulatory agency, has forced speculative traders out of soybean and silver markets because these limits were violated and prices were distorted.

Trading by Commodity Index Funds

Index funds in many respects look like speculative traders.  They do not use the physical commodity and have no physical commodity market transactions to coordinate with futures transactions.  The rationale for their investments in futures is to protect their investments through a hedge against inflation.  Through an unusual set of transactions, CFTC appears to have deviated from its traditional definition of hedgers to classify Index funds as hedgers rather than speculators.  The hedger classification frees them from speculative position limits and also provides them with lower margin deposit requirements for futures purchases than would be typical for speculators.  Margin deposits are somewhat like down payments on the contracts that are purchased or initially sold short.  With a small initial margin deposit, traders can buy a large volume of futures contracts.  CFTC Regulation 1.3(z), 17 CFR 1.3(z).

Through an unusual set of transactions, CFTC has classified Index funds as hedgers rather than speculators.

The CFTC definition of hedging is shown below:

“A hedge is a derivative transaction or position that represents a substitute for transactions or positions to be taken at a later time in a physical marketing channel.
Hedges must reduce risk for a commercial enterprise and must arise from a change in the value of the hedger's (current or anticipated) assets or liabilities. For example, a short hedge includes sales for future delivery (short futures positions) that do not exceed its physical exposure in the commodity in terms of inventory, fixed-price purchases and anticipated production over the next 12 months. The Commission and exchanges grant exemptions to their position limits for bona fide hedging, as defined in CFTC Regulation 1.3(z) “

Companies making index fund investments go through swap transactions outside the futures market.  Commodity futures brokers buy futures and then re-sell them to the index funds through these outside transactions, thus creating the buy-sell combination that CFTC has used as a rationale for the hedging classification.  But unlike traditional hedging, these buy-sell transactions are not linked to the physical commodities.  Some have described this process as being the equivalent of traders buying up large amounts of the commodity and holding it for long periods of time. 
For the past seven years, these “inflation hedges” have worked very effectively.  But, unlike a commodity hedge, there is no underlying mechanism that assures they will work well in the future.  As prices have moved higher, these positions have become very profitable and have drawn more money into index hedge funds.  Goldman-Sach is a company offering a commodity investment fund of this type.  At the end of November 2001, its commodity index stood at 168.7.  In late June 2008, this index was trading at 866.9, an increase of 414 percent in slightly over six and one-half years.  In just the seventeen months ending in late June 2008, the index  slightly more than doubled.  A chart showing these values is available at this website.

These are phenomenal rates of return compared with those available in stocks and bonds over the same time period.  As commodity prices have risen, index funds have become increasingly attractive and have tended to draw additional money into these positions.  However, as this is being written, some of the index funds have started to liquidate their commodity holdings, thus contributing to downward pressure on commodity prices.

Market Impacts

There are views on both sides of the argument of whether index fund traders have been a significant factor behind the sharp increase in commodity prices in the last few years.  Up to mid-July 2008, corn and soybean prices do not look out of line with energy prices.  However, the crude oil market is a key one. It is difficult to obtain clear-cut bench-marks to measure crude oil futures prices against in addressing whether index funds have contributed to increased oil prices.  Crude oil prices are the driving force in the rapidly expanding biofuels industry, and these funds appear to be concentrated more heavily in energy futures than in agricultural commodities.  Roach Ag. Marketing, Ltd., in its July 25, 2008, Sell Signals Newsletter, p. 1 indicated the DJ-AIG commodity index futures, which is used by fund traders, is composed of a portfolio of 37.6% energy futures; 19.54% base metals; 16.57% grains; 8.9% precious metals; 7.59% soft commodities; 6.97% livestock; and 2.74% soybean oil.

CFTC provides data on the volume of trading done in agricultural commodities by these funds, but not in the energy markets.  Some analysts indicate large fund traders have similarities to individual speculators in the past who accumulated large positions in commodity futures to corner the market and profit from resulting high prices.

Index hedge fund positions typically are in near-by and/or a limited number of distant futures contract delivery months.  When in near-by futures, they are rolled to a more distant contract before reaching the delivery period.

In the futures markets, each purchase of a contract requires an offsetting sale by someone else.  Sales have been generated to offset this rapidly expanding volume of purchases, but sharply higher prices have been required to attract sellers to the market.

Congress and the CFTC held hearings this spring in an effort to understand how index investment funds may impact the market.  As this is being written, both houses of Congress are working on legislation that may apply speculative trading limits to these traders and could require more complete CFTC reporting on their activities and volume of trading.

Conclusions: Index Funds vs. Speculators & Market Impacts

Index hedge funds are different than traditional speculative traders in that they typically are on the buy side of the market rather than the sell side, and because they focus on holding those long (purchased) positions for a long period of time.  Conversely, other speculative traders may shift from long purchases to short sales within a matter of minutes, hours, days, or weeks.  They typically trade on both sides of the market rather than just on the buy side.  Their rapid shift in direction of trades gives liquidity to the market, which is a very different role than that of traders who concentrate only on the buy side.  Index funds also are very different from traditional hedgers in that they do not coordinate futures transactions with physical commodity transactions.

Over the longer term, supply and demand factors will direct energy prices.  That in turn will be one of several important influences on prices of crude oil, ethanol, biodiesel, corn, soybeans, and other crops..  Recent high prices, whether caused entirely by global supply-demand conditions or partly by the large inflow of money into long futures positions, is stimulating increased investment in oil exploration and drilling in areas where environmental restrictions do not prevent that activity.

Next month we will look at that dimension of the energy market.  Those same high prices are a major catalyst behind the surge of investments into biofuels and other alternative sources of energy, as well as investments to develop more efficient motor vehicles. But the short-term (possible multi-year) impacts of index funds are less clear.  If these funds are rapidly forced out of the market or forced to downsize their positions to requirements of typical speculators, significant short-term downward pressure on prices is probable.  If fundamental supply-demand factors are the primary drivers of high oil prices, that effect would be temporary.


Testimony of Michael W. Masters, Managing Member / Portfolio Manager Masters, Capital Management, LLC, before the Committee on Homeland Security and Governmental Affairs United States Senate, May 20, 2008.

U.S. Commodity Futures Trading Commission. Speculative Limits, Accessed August 14, 2008.

Testimony of The Commodity Markets Council On Financial Speculation in Commodity Markets: Are Institutional Investors and Hedge Funds Contributing To Food and Energy Price Inflation? Before the Committee on Homeland Security and Governmental Affairs U.S. Senate, May 20, 2008.