Redefinition of the Scarcity Price : lessons and recommendations from the Colombian electricity market

The Colombian government introduced a capacity market in order to promote the diversification in the energy matrix and protect users from high prices derived from dry seasonal events. Unfortunately, the flaws in the scarcity price definitiona mechanism that activates the capacity market obligation and sets a cap price for the spot markethave led into a market failure. Specifically, some generation plants have been forced to be unavailable because their variable costs are significantly higher than the scarcity price. This paper presents a qualitative and quantitative analysis of the Colombian Electricity system, with a particular emphasis on the definition of the scarcity price. Results present lessons and recommendations for policy makers based on the experience of the Colombian Electricity Market, highlighting the need of a new definition of the scarcity price and a different focus on the energy planning scheme.


INTRODUCTION
RIGGERED by a supply shortage provoked by an intense dry season, Colombian Electrical sector started its a deregulation process during mid-90's [1].A new Electrical market structure was introduced by the change in the regulatory framework.There were two main objectives of this reform: stabilize the market, and increase the system reliability.The market structure evolved from a regional vertical integrated companies owned by the state to an inclusion of the private sector and a division of companies into four main activities: generation, transmission, distribution, and retail [2].
Electricity markets in South America gained broad experience of the process of deregulation [3].In Colombia, the deregulation of electricity started in 1994, and the spot market initiated operations in July 1995, supported by Laws 142 and 143.Despite some technological similarities with the Chilean electricity system, Colombia adapted the British model in the mid-1990s.Therefore, the Colombia's wholesale electricity market-Mercado de Energía Mayorista or MEM-is the only one in the region where pool prices are settled in a bidding process [4].
Four main transactions occur in the MEM: i) Transactions in the spot market, where retailers buy the energy they need to meet the requirements of their demand, ii) Bilateral financial contracts between retailers and generators, which aim to reduce the agent's exposure to market risk, iii) Auctions to allocate the Firm Energy Obligations -Obligaciones de Energía Firme or OEFs-under the scheme known as the Reliability Charge-Cargo por Confiabilidad or RC and iv) Other Services to provide ancillary power generation services, such as the automatic generation control (AGC) [5].

A. Allocation of Firm Energy Obligations in the MEM
In terms of the Colombia's installed capacity, approximately 64% of it comes from hydro-generation technologies and a minor proportion from thermal-generation plants (31%) [6].This dependency on hydraulic resources causes that Colombia is particularly sensitive to a phenomenon known as "El Niño", characterized by an intensive dry season with an approximate duration of 1 year [7].As a result, it has been almost imperative for the electric energy sector to have sufficient thermal resources and hydro reservoirs with firm energy to replace hydro-generated energy in dry periods.
Therefore, in 2006, the Colombian Commission for the Regulation of Energy and Gas (CREG) introduced a new scheme to guarantee the availability of sufficient capacity to meet peak demand during long dry seasons and to ensure the long-term reliability of the electricity supply in Colombia [8].The scheme allocates Firm Energy Obligations to new and existing generation plants at price determined in competitive auctions.The OEFs are "option contracts" that commit generators to supply given amounts of energy at a predetermined Scarcity Price (SP), during scarcity situations.In return for agreeing to supply at the SP, during the commitment period, generators allocated with OEFs receive a fixed annual option fee for each kilowatt hour (kWh) contracted.This fee is called the Reliability Charge.It is paid by consumers in their electricity tariff and is determined in the auction in which the generator sold its firm energy.Generators with OEFs receive the RC regardless of their actual dispatch and whether the fulfilment of their obligation is required or not.
The main benefit of this charge is the contribution to recover a portion of the fixed costs for generation agents, especially for peaking power plants, which does not deliver energy to the grid frequently during regular weather conditions [8].
In case the energy generated is more than the obligation specified in the OEF, this additional energy will be paid or rewarded at the spot market price [9].If generators cannot meet their OEFs, they pay a penalty, equal to the difference between the spot market price and the scarcity price on the OEF quantity not met in any hour.

B. Remuneration of the OEF: definition of the Scarcity Price
Scarcity pricing is a regulatory mechanism used by electrical markets to establish a cap market price sufficiently high during low system reliability periods, before the demand is not covered due to a shortage in supply [10].Properly adjusted SP would help to incentivize: demand reduction, higher generation availability and additional energy imports.With a consistent increase of renewable generation in terms of installed capacity globally, Colombia appears as a suitable referent for current and future Electrical markets with a predominant renewable energy matrix and a possible exposure of scarcity of resources.In this sense, it is important to explain and recommend solutions for the current flaws of the scarcity price approach in the Colombian Electrical market.This experience can be valuable in order to establish more robust structures for markets with a high penetration of renewable generation.
By definition, in Colombia the scarcity situation established when the Spot Market Price (SPM) surpasses the SP [9].It is established by the CREG and updated on a monthly basis with the variation of the fuel operational costs of the most inefficient power plant, and other variable costs (OVCs) that generators have to pay to the system.Equations ( 1) to (6) show the methodology to calculate the SP [11]: Where:   : Scarcity Price for the month m in COP $/kWh They are related to the National interconnected system (SIN) for the month of interest minus one, in COP $/kWh, and depend on: Where: EEC m : Energy equivalent cost for the month of interest in COP $/kWh AGC m−1 : Automatic gain control cost for the month of interest minus one, in COP $/kWh FAZNI m−1 : Fund for the non-interconnected zones of the Country for the month of interest minus one, in COP $/kWh Law99 m−1 : Monetary contribution of the Law 99-1993 for the thermal plants for the month of interest minus one, in COP $/kWh

Operation and Maintenance Costs (𝑶𝑴𝑪 𝒎−𝟏 ):
They are calculated for the month of interest minus one in COP $/kWh.This value depends on: Where As an example, in September 2015, XM published the estimated value of the SP for October 2015 [12].The calculation is based on the CREG resolution No. 070-2014.
Table I shows a detailed explanation on the calculation of this value, using (1) to (6).The obtained value for the SP was about 0.0977 USD/kWh.Despite this scheme designed to provide reliability in the Colombia's hydro-dominated electricity market, current electricity spot market prices in the MEM are increasing more than tenfold, setting an energy crisis in Colombia (See Fig. 1).
In consequence, the generation plants are paid with the SP which is lower than the variable cost of the generation plant that covers the demand on scarcity or peak periods-it means the plant with the highest variable cost.Thus, many generation plants are forced to be unavailable as a result of the extortionate costs of generating at a significantly higher cost than the SP.The prices were converted into USD using the same ECR of SP t f (adapted from [13]).
In this sense, the current regulatory policy is not solving the problem of "missing money" (the amount of money per MW of capacity that a generator is missing in scarcity hours) [14].This situation is a consequence of: First, one of the strongest "El Niño" periods that the country has faced, and Second, a failure in the electricity regulatory framework, regarding the definition of the SP.
Considering the above evidence, we present an analysis of the Scarcity Price' definition, its main flaws, the measurements already taken by the Colombian government, some recommendations to overcome the crisis , and the lessons learnt from the Colombian Electricity Market, applicable to other similar markets.The recommendations address the need to include non-conventional energies (not affected by the dry seasons) to provide firm energy, and a new definition of the SP, considering its concept in terms of the generation plants costs and not in terms of the plants' technical efficiency.These amendments could apply to other countries, which electricity generation relies mainly on hydro or have a similar energy markets and dry climate seasons.

MARKETS
The relevance and the mechanism which scarcity of generation resources is dealt with, vary depending on the market structure.These approaches were categorized into two main market structures: Energy only markets and Energy markets with capacity payments.

A. Energy Only Markets
By definition, in Energy only markets generators agents can only obtain revenues through spot market and ancillary services [15].This market design leads to moderate prices during regular conditions and, occasional high prices during scarcity conditions.In this context, setting adequate scarcity price levels is crucial for a correct market performance.If the scarcity price is too high, excessive generating capacity will be built as a result, thus more power plants would not be operating during regular conditions, increasing the market exposure for the missing money problem.In contrast, if scarcity price is too low, there will be no incentive to build new generation capacity, increasing the possibility for future energy shortages [16].Some scarcity pricing approaches had performed problems in Energy only markets.In the Electric Reliability Council of Texas (ERCOT) and Australia's National Electrical Market (NEM), the SPOE is set by the highest bid price offer of generator agents, during normal conditions.In this process, the bids of large generator agents are mitigated; the SP is mainly established by small generator agents [17].There are two main problems with this SPOE approach.Firstly, SPOE calculation relies only on some groups of generator agents to accurately predict the existence and the magnitude of a scarcity period, during normal conditions [18].Second, according to [19], price bids of small-unmitigated generators agents vary widely under identical system condition, and as a result, there is a

B. Energy Markets with capacity payments
Although scarcity situations are necessary less frequent in Energy market with an integration of capacity payments, they cannot be avoided completely.Scarcity pricing appears as a suitable approach for a better market performance [3].The benefits of a proper Scarcity pricing approach are just not limited to send investment signal.Despite the design of the energy market, through a correct definition and method of SPOE calculation, resources can be allocated more efficiently and incentivise a higher demand response [20].
In the New England energy market (ISO-NE) the SPOE is defined as the maximum cost incurred to meet the reserve requirements-also known as Reserve Constraint Penalty Factor (RCPF)-, and the scarcity situation is defined when the SP is higher than RCPF.The RCPF set a cap for the sport market; if the RCPF is lower than the cost of meeting the reserve requirement level, these requirements will not be met (ISO-NE, 2009).Unlike the Colombian approach, during scarcity situations, in the ISO-NE, market scarcity revenues are subtracted from the capacity payments, avoiding extra revenues for agents.
Despite some energy markets do not have a specific mechanism for scarcity pricing, they have established markets structures to deal with scarcity periods.For example, South Korea and Chile Energy markets have a capacity market based on availability.With a fast paced demand growth, energy security is a priority for these countries.Therefore, the capacity payment must be high enough to attract the investment in order to avoid energy rationing [3].The former Colombian capacity market was designed based on the Chilean initial design, where capacity payments are made based on the installed capacity of the power plant, and the generator availability during peak demands months or capacity shortage situation [21].Moreover, in the South Korea capacity market structure, the payment is different for base-load and peaking generation units.This payment is based on the gross capital and the O&M cost of a Coal and Gas generation unit, for base-load and peaking power plants, respectively [22].

III. FAILURES OF THE SCARCITY PRICE DEFINITION IN COLOMBIA
There were two main objectives with the introduction of the SP regulation.First, avoid the peak prices that the demand has to pay in scarcity periods.Second, promote a good investment climate by a continuous remuneration to the Electrical Generators agents.It would guarantee the energy security of the country in terms of supply, especially due to a "gas crush", motivating most of thermal generation power plants to change their fuel supply from gas to fuel oil based [9].
This section analyses the reasons why the SP is not well defined and describes the basic rationale behind the problem.

A. Main flaws in the definition of the Scarcity price
The SP should be based on the variable cost of the peak power plant that attends the demand in the scarcity [23].Instead, in the Colombian market the SP is defined based on the variable costs of the most "technically inefficient" plant, which is not always the plant that attends the peak demand.Other plants with higher technical efficiencies but also with higher variable costs are the ones who attend the peaks hours.
With the latest spot and scarcity prices, the Colombian government took over a power generation agent, Termocandelaria (314 MW of installed capacity), because it was not generating energy due to the huge debt provoked by the difference between the SP and the variable generation cost of this agent [24].
This situation is particularly critical for other generation agents, and for the electricity market as a whole.There is a big probability that other generation agents could replicate the current situation of Termocandelaria, attempting directly to the Energy security of the country.Mainly, because "El Niño" is a seasonal event that impacts Colombia's raining every 2 to 6 years approximately [25] and also, because the current definition of the SP is heavily sensible to the international oil spot price.The SP's formula has a strong correlation with oil prices (linked to the fuel oil No. 6 index).But the reality is that most Colombian thermal plants are backed by Fuel oil No. 2, which is more expensive and it's less correlated with oil.Therefore, the current scarcity price's value comes from using the Fuel Oil No. 6 as a reference, because it's much cheaper than the diesel used by plants (fuel oil No. 2).Changing the SP represents a dilemma.On one hand, the CREG argues that a change from FON6 to FON2 in the SP definition would represent an increase in the SP regardless the time of the year.Therefore during normal weather conditions, agents have an incentive to increase SPM because it would represent a potential higher revenue.This situation would be reflected in higher electricity bills for consumers.On the other hand, the cap price at which thermal plants can sell is not properly adjusted, increasing the probability of financial unavailability of these plants in scarcity periods, jeopardising the energy security of the country [26].

B. Effects on the Market's agents: Generators and Consumers
During El Niño" period, the generation plants (especially thermal plants available in dry seasons) have increased their generation price and the final energy prices have also increased, affecting the final consumers' economy.
Since September 2015, with the Resolution No.178 of 2015, the CREG agreed to raise (for six months) the SP from COP $302 to COP $470.This measure applied only for thermal plants that generate with diesel, in order to overcome the scarcity period.Since then, the electricity tariff for residential consumers has increased by about 17.54 % [27].Although consumers are protected by the SP, if it increases, the maximum price that the demand has to assume also increases.
Nevertheless, the deadline for this measure is until April 28.
According to the Association of the largest consumers of electric energy in Colombia (ASOENERGIA), they are not willing to accept any rise in the electricity tariff, after this deadline.The guild stresses the need to adopt measures to correct the structural problems affecting the power system [28].
From the point of view of the generators, due to the Millionaire losses they are facing because of the crisis, they are asking for a new agreement.According to [29], operators of the 12 thermal generation plants (which use gas and diesel) are willing to deliver their companies to the government, if by May 1, 2016 the CREG does not adjust the scarcity price to more than double.

IV. ACTIONS TAKEN BY THE GOVERNMENT
The Colombian government had taken some actions to avoid an energy crisis that could lead a possible blackout of the country in a certain hour of the day.Specifically, here we critique the rationale and effects of three measures taken by the Colombian government: the CREG regulation 172 and 178 of 2015, the energy save program "Apagar Paga" and other measures also launched by the government.

A. CREG resolution No. 172 of 2015
Given the peaks prices of the SPM, almost 7 times greater than the SP (see Fig 1), the CREG introduced a cap price in the day-ahead bid process by publishing the CREG resolution No. 172 of 2015 [30].The cap price is equal to the 75% of the first level of Incremental Operational Cost of Energy Blackout, (CRO1).Currently the CRO1 is equal to COP $ 1191.07 [31]; therefore the cap price of the spot market of electricity in Colombia is COP $893.31.However this cap price only applies when the variable HSIN (Hydrology of the National Interconnected System) is lower than 90% or when the CREG determine that is necessary.The variable HSIN measure the amount of energy produced from hydraulic resources compared to the previous month.
Despite the effectiveness of this measure, there are also some downsides.First the "missing money" problem is not solved.Thermal power plant with energy firm obligation will still be paid at the SP.Second; the 75% of the CRO1 is still lower than the cost of the peaking power plant.It means that some thermal power plants will not recover their variable cost even if they sell their energy at the SMP.Second, the time of the Penalty Exposure remains the same.Although, the cap price for the day-ahead bid process reduces the magnitude of the penalty exposure of the agents, the SPM still higher than the SP level; therefore the time frame of the scarcity situation has not changed.
Additionally, the price cap introduced in the regulation just depends on the level of the variable HSIN to be activated.This variable implies a comparison of at least two months of generation coming from hydro resources.

B. CREG resolution No. 178 of 2015
The CREG decided to establish a temporary floor for six month to the scarcity price.Through the CREG resolution 178 of 2015, the institution established that the SP would be the greater between, its monthly update and the SP of October 2015 [32].Moreover, for the generation agents that use fuel oil, the remuneration of the firm energy obligation will be approximately COP $470, which is 1.5 times higher than the current SP (approximately COP $302).This regulation also helps to reduce the time of penalty exposure of the agents.However, just like the CREG Resolution No. 172, the focus is more to reduce the magnitude of the penalty exposure and not the time.The resolution does not solve the "missing money" problem either.It is true that in theory there is lower probability of the event where the SMP would be greater than the SP.Nevertheless, given the current water reservoir level and the El Niño/Southern Oscillation (ENSO) forecast [33] and the recent SPM levels (See Fig. 1), it is highly possible that the SMP will be greater than the October SP level.
Moreover, the resolution is a transitory measure and implies different market conditions for certain agents.Regardless the effectiveness of this resolution, the scope is only valid until May 2016, and according to the forecast, the dry season will remain until August of 2016.

C. Energy save programme "Apagar paga"
Driven by the current intense and long dry season, and the high possibility of an energy black out, the Colombian government launched an energy save programme called "Apagar paga" or "Turn off, pay off" (unofficial translation).The rationale is simple: promote the energy daily saving of at least 5% in order to avoid possible blackouts [34].
Behind the widely spread advertising camping through national TV or social media, the programme includes a set of penalty for the users that do not meet the energy savaging targets.Through the CREG Resolution No. 029 of 2016, for each extra kWh compared to the consumption in February of 2015, the users have to pay an additional amount of COP $450.This particular feature of the programme has been polemical.On one hand, the government argues that there must be a price mechanism to incentive the energy saving in the country.On the other hand, normal users argue that the government did not take the correct measure and, now they have to be overcharge for a market failure problem [35], [36].
In addition, the programme was launched late.Since September of 2015, the system has been in a "scarcity situation" and the water reservoir level has dramatically dropped since then.In Contrast, the energy saving programme was launched in March of 2016, forcing to a dramatic change in behaviour.It is possible that the saving targets are met.However, in order to truly change people behaviour and move them towards a more efficient use of energy, it is necessary the implementation of long-term programmes [37], [38], not just an immediate implementation used as a "safety valve" measure.

D. Other short-term measures
Since it seems that "El Nino" period is testing the Colombia's electricity framework, the government has had to enact these other temporary measures to preserve the viability of the system: 1) Increasing the gas supply for thermal generation plants.However, current gas supply of the country is limited and the amount of gas required by the thermo-electric sector is too high.Therefore, it is expected that the regasification plant of liquid natural gas (LNG) will start operation by the end of 2016, a few months later of the projected end of the dry season.
2) Importing additional energy from Ecuador.Since December 2015 Colombia has imported energy from Ecuador, reaching to a peak in March 2016 of about 5.6 GWh.This measure has helped to reduce the stress of the system, although it represents a minor fraction of the country's electricity demand (5522 GWh) [6], [39].

A. The need for a better energy planning
El Niño has left in doubt the Colombia's hydroelectric capacity to meet the whole electricity demand, which is increasing as a result of the economic activity and the population growth.All governments in Latin America know that unusually dry periods appear, roughly once every 10 years [40] and therefore, there is no reason that justifies why Colombia has the possibility of going to an electrical collapse, because it was already known that "El Niño" would return.
According to the Colombia's Generation and Transmission Expansion Plan 2006-2020, the electricity demand is expected to continue increasing by between 2.8 % and 3.8 % per year in the "Medium" scenario.This increase together with the phasing out of existing power plants (when they reach their end of lifetime), means that there will be a need for establishing new power generation facilities [8].
Hence, Colombia should consider nonconventional renewable sources of firm energy including wind, solar, biomass and geothermal, such as alternatives to thermal plants.Nevertheless, the promotion of an efficient mix of resources and the achievement of a firm energy market that provides reliable electricity at least cost, requires that all resources, including variable resources such as wind power, should be eligible to receive the same reliability payment, based upon the resources 'ability to provide firm energy [40].
The main benefits of this measure would be: 1) it leads to a more efficient mix of resources and reduces risk by establishing a more diversified portfolio instead of fossil fuels, which also could reduce electricity costs, 2) the long lead times required for large hydro and coal plants contrast with relatively limited externalities and flexibility offered by nonconventional renewable sources of energy, 3) they do not depend on the fossil fuel's price volatility, and 4) it reduces Colombia's reliance on coal and other fossil fuels to generate electricity during dry periods, hence reducing Colombia's emissions from fossil fuels and the negative effect on the environment and human health.4) Likewise, with the current COP21 agreement there is an expected increase share of renewable energies, especially in the electricity sector [41].A greater introduction of renewable technology in the generation park would help to achieve the international environmental target.
In particular, the wind regime in Colombia is among the best in South America.The potential for wind power is 18 GW, which is 900 times as much as the current capacity of 20 MW [42].Unfortunately, according to [39], the RC commitment only considers hydro and thermal power plants' projects.From the 13 approved projects to entry the market until 2020, 69.23% of them corresponds to hydro power plants.It represents a total installed capacity of about 3769.MW out of 4431.4MW, the total projected energy firm that will be provided by the projects.Thus, it means that none amendments are prevised for a future presence of "El Niño" period.
In addition, in the last firm energy auction (in 2011), wind power was not eligible for firm energy payment.The main reason is the lack of information about the resource's ability to supply firm energy.This is already done for hydro and thermal resources, but there is a need of an analogous methodology to estimate firm energy for variable resources [40].

B. New definition of the Scarcity Price: Case Study
Besides the above efforts made by the government, we define another possible proposal that could fix the current problems: Taking into account that the current SP does not reflect the actual situation of the market, it is proposed to use the heat rate and the fuel of the most expensive power plant in the market as a reference to calculate the variable cost (instead of the most inefficient one).The intention of this measure is to cover the variable costs of peak power plants and by this avoid a possible market collapse.
We simulate the behaviour of the scarcity price by considering the variable cost of the most expensive plant in the market.It corresponds to the thermal plant "Termocandelaria", which has a heat rate of 10 MBTU/kWh.Although it is the most expensive plant, it is not the most inefficient.In terms of efficiency (heat rate), the value varies between 6 MBTU/kWh and 12 MBTU/kWh and other thermal plants such as "Termoflores 2" (Diesel) and "Termocartagena 2" (Fuel Oil), have values of about 12 MBTU/kWh [43].
Together with the above assumptions, we use Diesel (Fuel No.2 instead of Fuel No.6) as the operation fuel and the values for other external costs (such as some taxes and other legal deductions) remain the same.Also we considered the average Spot Market Price to calculate both, the time and magnitude of penalty exposure.
We applied an analysis backwards in time, in a scope of one year (from April 2015 to March 2016), using (1) to (6).Fig. 2 shows a new estimation of the scarcity price contrasted with the current scarcity price and the spot market price.The new definition of the SP brings the following benefits: 1) it solves the "missing money problem", because the plants are paid according to the most expensive plant, therefore if the variable cost of the peak plants is covered, also it does for the rest of the plants.2) It reduces the time of penalty exposure in 60%, since the number of times the plants are exposed to market (because of the scarcity price) reduces from 196 times (using current SP) to 79 (using new definition of the SP) and 3) it also reduces the magnitude of the penalty exposure in 77%.It means the variation in price ($/kWh) during scarcity periods.

VI. CONCLUSIONS
"El Niño" period has put to test the Colombian Energy Capacity Market, and the regulatory framework as a whole.The combined situation of high spot market prices and low scarcity prices has affected both, Generations agents and consumers.The current scarcity price definition is based on the cost of the most technical inefficient power plant in the market, not covering the variable cost of more efficient yet more expensive plants.Consequently, some generation agents have been financially unable to operate in the market due to the big debt acquired during the scarcity period.
Despite the relative effectiveness of the measures taken by the government to mitigate the effects of the current energy crisis, we suggest three main amendments: redefining the scarcity price in terms of the plant with the highest variable costs of the market; redirecting the focus of the energy expansion plan through a more diversified energy matrix, centred in the inclusion of alternative renewable energy sources; and enabling long term energy efficiency programmes.Results show that the proposed scarcity price definition mitigates the agent's penalty exposure in both dimensions: magnitude and time.However, since the new value of the scarcity price is higher compared to the current levels, the consumers could be potentially exposed to higher energy tariffs due to the increase of the spot market cap price.Moreover, considering the country's high potential of solar, wind and geothermal energy, these technologies could provide firm energy, but it requires mainly two conditions: First, more studies with accurate and updated data.Second, adjustments in the regulatory framework of the Colombian electricity market to support their inclusion.Furthermore, aligned with the measures mentioned above, long-term programmes should be implemented to truly change people's behaviour and move them towards to a more efficient use of energy.

Fig. 1 .
Fig.1.Comparison between the Scarcity Price and the Spot Market Price in Colombia, 2015.Note: The prices were converted into USD using the same ECR of SP t f (adapted from[13]).
vs Scarcity Price SPM-Spot Market Price USD SP-Scarcity Price USD wide range of SPOE calculated under similar conditions.

Fig. 2 .
Fig. 2. Proposal for a new definition of the SP: Case Study.Note: The Scarcity prices were converted into USD using the same ECR of SP t f .
Market Price USD SP-Scarcity Price USD PSP-Proposed Scarcity Price USD