Evaluation of electricity theft detection sensors in intelligent networks
In many countries, theft of electricity is estimated at billions of dollars annually. In order to reduce the volume of electricity theft, public power plants optimize data obtained by the new Developed Measurement Infrastructure (IIM), and use their analysis to establish deviating rates of consumption and possible fraud. In this study, we propose high-priority strategies for using data analysis to establish cases of theft of electricity and new measurement principles that will help optimize these strategies for evaluating and comparing anomalous indicators. We use real-life information from the RII system to substantiate our methodology.
Intelligent network is the modernization of the infrastructure of the power system with the help of new technologies, which allows the automatic system to demonstrate greater efficiency, reliability and security, while at the same time providing greater transparency and choice for consumers. One of the key technologies that are actively involved around the world is the Technology of the Developed Measurement Infrastructure (IIM).
RII is a modernized measurement system, where old mechanical devices are replaced by new “smart” meters. Such devices are newly introduced devices that provide two-way communication between energy companies and consumers, as a result of which there is no need to send a service employee to take indicators, and new opportunities appear, such as the ability to monitor the amount of electricity consumed in more detail, and to detect failures more quickly with analogue ones. power company’s devices learned about failures mainly through consumer complaints — to automatically restore power supply, remote disconnection and sending information to consumers (for example, about price changes or renewable energy sources), provide consumers with more available information about their energy consumption.
“Smart” counters are billions of consumer devices with an operational resource of several decades and operating in physically unsafe places [16]. Complicating these devices by adding processors and protected memory can increase the price of smart meters by a few dollars, and since energy companies have to use millions of devices, the reality of the market is that in practice they are uneconomic and not recommended as necessary [21].
')
Therefore, some security measures have been developed (print control opening, secure connections), which at the same time was not enough to prevent intrusions during the life of the meter. In addition to the vulnerabilities identified by security experts [17,9] - it happened that scammers were updating embedded programs [20] - the hacked counters allowed electricity to be stolen, which cost the American energy company hundreds of millions of dollars annually, as reported by the FBI cyber intelligence report [14]. The FBI report warns that company employees and individuals with a minimal level of PC users are fully capable of opening and reprogramming the counter using inexpensive tools and programs that are easily accessible on the Internet. The FBI report also states with great certainty that as the use of intelligent networks is becoming more common in the country, such fraud will become more frequent due to the ease of access to the system and the financial gain for both the burglar and the consumer of electricity.
Detection of theft of electricity has always occurred through inspection of the opening control seals or balance meters [10]. Despite all the benefits, these methods are not enough. Autopsy seals can be easily removed [5], and balance meters can show that some of the users are using electricity incorrectly, but cannot show who it is. Despite some of the vulnerabilities of smart meters, the high resolution data they collect seems to be a promising technology to detect electricity theft.
In general, energy companies are gathering more information from a large number of devices to optimize the analysis of large amounts of data [15] in order to get a better knowledge of the state of their system. One of the key services provided by Measurement Data Management (MDM) is designed to transform a large amount of data into information of practical value and is called revenue security. This service implies the following: energy companies use software to analyze data on specific data on meter readings in order to detect possible incidents of electricity theft and readings that deviate from the norm [13]. Thus, analyzing a large amount of data becomes a new cost-effective way of supplementing the readings of the balance meters (which becomes necessary as fraudsters get access directly to the electricity distribution lines rather than break the meter) and direct intervention by an employee of the company who checks the integrity of the opening control print.
In this paper, we will focus on the problem of analyzing data from an MDM system to detect illegal use of electricity. Despite the fact that some MDM providers offer this service, their methods and action algorithms do not have public access, so it is not possible to evaluate the effectiveness of these methods. In addition, the few available studies on this topic have their drawbacks [18,19,11,6]: 1) They do not take into account the so-called threat model, and therefore it is not clear how the theft detection algorithm will cope with more sophisticated fraudsters, 2) their information has low resolution, hence they turn to non-parametric statistics, 3) they recognize that the data set of examples of fraud checks the accuracy of the classifiers, and therefore the evaluation of the data will be biased, since it will depend on how easily to establish fraud, having available data and unrecorded cases do not allow objectively evaluate the effectiveness of the classifier.
In this paper, we approve the following provisions: 1) We represent a potential hacker of the DM system. In previous works, the possibility of the existence of an “advanced” cracker was never taken into account. This threat model is especially important for digital counters, since an attacker who has access to an opened counter can send an arbitrarily selected fine-modular signal with an accuracy that was impossible with mechanical breaking (for example, using a strong magnet). 2) We offer a new system for measuring the accuracy of classification of abnormal indicators. This new metric system takes into account some of the main problems in detecting anomalous indicators in light of the safety issue: (a) the fact that the hacking cases recorded in the data set may not be characteristic of future hacks (therefore, a classifier able to fix these cases may do not recognize new more “advanced” hacking technologies), (b) in many cases it is rather difficult to obtain data on hacks for academic research - this is especially true for reducing s Integrated automated dispatch control system (CAS control unit) and the sensor data and actuator in the industrial and energy systems - that's why we say that we need to abandon the analysis and evaluation of classifiers with an unbalanced and irrelevant data set. (3) Using real-life IIM (data from fifteen-minute intervals collected over 6 months from 108 users) provided by the power company, we evaluate ways to detect electricity theft, including a new ARMA-GRL sensor created to capture the characteristic cases of illegal use of electricity (reducing electricity bills) in the formal model of a complex hypothesis.
In this section, we describe new common ways to evaluate the performance of classifiers in a hazardous environment. Since this concept can be used to solve other problems, we propose a model within the framework of a general classification. Our attention will be mainly focused on two topics: (1) the classification of hazardous conditions or how to evaluate the efficiency of the classifier in the case when a burglar performs an undetectable burglary, and (2) study dangerous situations or how to prevent the burglar from entering incorrect data.
In self-learning machines, classifiers are traditionally evaluated on the basis of test data containing negative (normal) and positive indicators (tampering, hacking). However, in dangerous conditions, there are many situations where it is impossible to obtain in advance examples of cases of hacking. Two reasons can be distinguished here: (1) we cannot by definition get data of hacks on “zero days”, (2) using examples of hacks fixed separately from the classifier means that the hacker does not adapt to existing conditions and will not try to avoid our detection mechanism.
In this study, we argue that instead of using a number of hacking examples to evaluate the classifier's performance, we should determine the worst possible hacking option for each classifier and evaluate it, taking into account the financial damage caused by this hacking option.
Models and assumptions:
We will model the problem of evaluating classifiers, presenting the worst hacking options as follows:
1. A random process generates an observation x ∈ X. These observations are the realization of a random vector X with the distribution P0.
2. Suppose that x can only be detected using a touch sensor (that is, a smart counter), and the touch sensor sends y to the classifier. Thus, while everyone knows P0, a specific example of x is known only to the touch sensor.
3. The sensor can be in two states (1) normal and (2) abnormal. If the sensor is in normal condition, then y = x. If anomalous, then y = h (x), where h: X → X is such a function for which the permissible distribution P1 for Y satisfies the Attitude (cracker's intentions): g (X) R g (Y) (that is, E [Y ] <E [X] where E [X] denotes the expected variable X).
4. The classifier f: X → {n, p} outputs the solution: negative n when deducing that y is an example of P0 and positive p if y belongs to P1.
Metric system for evaluating classifiers in a hazardous environment:
To simulate hacking, we introduce the cost function C (xi, yi), which forms the vector yi, modifying the original value xi in such a way that yi denotes an example of hacking, which gives the maximum value C (xi, yi) if it remains undetected. Namely, we have the following:
1. The data set N = {x1, ..., xm} ∈ X m where each xi is considered an example from the set P0. Note that xi ∈ X. A common example is X = Rd, that is, every observation xi is a vector of real-life values with dimension d. When using a smart meter, this may mean that I xi refers to meter readings recorded in 24 hours.
2. The value α∈ [0, 1] representing the upper permissible limit of a false alarm is in the data set N.
3. The cost function C: X × X → R denotes the cost of damage from a false negative indicator.
4. A data set of one of the classifiers F = {f0, ..., fq}, where each classifier has parameters in the limit τ used for decision making. If we want to leave unknown the limit used by a separate classifier, we use the notation fi, τi.
This is only the beginning of the translated work.
The full text of the translated article can be downloaded here.
1. EWMA Control Charts, itl.nist.gov/div898/handbook/pmc/section3/pmc324.html
2. forecast package for R, robjhyndman.com/software/forecast
3. RapidMiner, rapid-i.com
4. Antmann, P .: Reducing technical and non-technical losses in the power sector. Technical report, World Bank (July 2009)
5. Appel, A .: Security seals on voting machines: A case study. ACM Transactions on Security Systems 14, 1–29 (2011)
6. Bandim, C., Alves Jr., J., Pinto Jr., A., Souza, F., Loureiro, M., Magalhaes, C., Galvez-Durand, F .: Identification of energy theft and tampered meters using a central observer meter: a mathematical approach. In: 2003 IEEE PES Transmission and Distribution Conference and Exposition, vol. 1, pp. 163–168. IEEE (2003)
7. Breunig, M., Kriegel, H.-P., Ng, RT, Sander, J .: Lof: Identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93-104. ACM (2000)
8. Brodsky, B., Darkhovsky, B. .: Non-Parametric Methods in Change-Point Problems. Kluwer Academic Publishers (1993)
9. Davis, M .: Smartgrid device security. adventures in a new medium (July 2009), www.blackhat.com/presentations/bh-usa-09/MDAVIS BHUSA09-Davis-AMI-SLIDES.pdf
10. De Buda, E .: System for accurately detecting electricity theft. US Patent Application 12/351978 (January 2010)
11. Depuru, S., Wang, L., Devabhaktuni, V .: Support theft. In: Power Systems Conference and Ex position (PSCE), 2011 IEEE / PES, pp. 1–8 (March 2011)
12. ECI Telecom. Fighting Electricity Theft with Advanced Metering Infrastructure (March 2011)
13. Geschickter, C .: The Emergence of Data Management (MDM): A Smart Grid Information Strategy Report. GTM Research (2010)
14. Krebs, B .: FBI: smart meter hacks likely to spread (April 2012), krebsonsecurity.com/2012/04/fbi-smart-meter-hacks-likely-to-spread
15. Lesser, A .: When it’s big IT goes on smart grid (March 2012), gigaom.com/cleantech/when-big-it-goes-after-big-data-on-the-smart-grid -2
16. McLaughlin, S., Podkuiko, D., McDaniel, P .: Energy Theft in the Advanced Environmental Infrastructure. In: Rome, E., Bloomfield, R. (eds.) CRITIS 2009. LNCS, vol. 6027, pp. 176–187. Springer, Heidelberg (2010)
17. McLaughlin, S., Podkuiko, D., Miadzvezhanka, S., Delozier, A., McDaniel, P .: Multi-vendor penetration testing. In: Pro-ceedings of the Annual Computer Security Applications Conference (ACSAC) (De-cember 2010)
18. Nagi, J., Yap, KS, Tiong, SK, Ahmed, SK, Mohamad, M .: Non-technical loss testing for power users using support vector machines. IEEE Transactions on Power Delivery Systems 25 (2), 1162–1171 (2010)
19. Nizar, A., Dong, Z .: Identification and customer of customer behavior irregularities. In: Power Systems Conference and Exposition (PSCE), pp. 1–10 (March 2009)
20. Peterson, D .: AppSecDC in review: Real-world backdoors on industrial devices (April 2012), www.digitalbond.com/2012/04/11/appsecdc-in-review
21. Smart Grid Interoperability Panel, editor. NISTIR 7628. Guidelines for Smart Grid Cyber Security. NIST (August 2010)
22. Sommer, R., Paxson, V .: Outside the world: On-line intrusion detection. In: IEEE Symposium on Security and Privacy (2010)
Intelligent network is the modernization of the infrastructure of the power system with the help of new technologies, which allows the automatic system to demonstrate greater efficiency, reliability and security, while at the same time providing greater transparency and choice for consumers. One of the key technologies that are actively involved around the world is the Technology of the Developed Measurement Infrastructure (IIM).
RII is a modernized measurement system, where old mechanical devices are replaced by new “smart” meters. Such devices are newly introduced devices that provide two-way communication between energy companies and consumers, as a result of which there is no need to send a service employee to take indicators, and new opportunities appear, such as the ability to monitor the amount of electricity consumed in more detail, and to detect failures more quickly with analogue ones. power company’s devices learned about failures mainly through consumer complaints — to automatically restore power supply, remote disconnection and sending information to consumers (for example, about price changes or renewable energy sources), provide consumers with more available information about their energy consumption.
“Smart” counters are billions of consumer devices with an operational resource of several decades and operating in physically unsafe places [16]. Complicating these devices by adding processors and protected memory can increase the price of smart meters by a few dollars, and since energy companies have to use millions of devices, the reality of the market is that in practice they are uneconomic and not recommended as necessary [21].
')
Therefore, some security measures have been developed (print control opening, secure connections), which at the same time was not enough to prevent intrusions during the life of the meter. In addition to the vulnerabilities identified by security experts [17,9] - it happened that scammers were updating embedded programs [20] - the hacked counters allowed electricity to be stolen, which cost the American energy company hundreds of millions of dollars annually, as reported by the FBI cyber intelligence report [14]. The FBI report warns that company employees and individuals with a minimal level of PC users are fully capable of opening and reprogramming the counter using inexpensive tools and programs that are easily accessible on the Internet. The FBI report also states with great certainty that as the use of intelligent networks is becoming more common in the country, such fraud will become more frequent due to the ease of access to the system and the financial gain for both the burglar and the consumer of electricity.
Detection of theft of electricity has always occurred through inspection of the opening control seals or balance meters [10]. Despite all the benefits, these methods are not enough. Autopsy seals can be easily removed [5], and balance meters can show that some of the users are using electricity incorrectly, but cannot show who it is. Despite some of the vulnerabilities of smart meters, the high resolution data they collect seems to be a promising technology to detect electricity theft.
In general, energy companies are gathering more information from a large number of devices to optimize the analysis of large amounts of data [15] in order to get a better knowledge of the state of their system. One of the key services provided by Measurement Data Management (MDM) is designed to transform a large amount of data into information of practical value and is called revenue security. This service implies the following: energy companies use software to analyze data on specific data on meter readings in order to detect possible incidents of electricity theft and readings that deviate from the norm [13]. Thus, analyzing a large amount of data becomes a new cost-effective way of supplementing the readings of the balance meters (which becomes necessary as fraudsters get access directly to the electricity distribution lines rather than break the meter) and direct intervention by an employee of the company who checks the integrity of the opening control print.
In this paper, we will focus on the problem of analyzing data from an MDM system to detect illegal use of electricity. Despite the fact that some MDM providers offer this service, their methods and action algorithms do not have public access, so it is not possible to evaluate the effectiveness of these methods. In addition, the few available studies on this topic have their drawbacks [18,19,11,6]: 1) They do not take into account the so-called threat model, and therefore it is not clear how the theft detection algorithm will cope with more sophisticated fraudsters, 2) their information has low resolution, hence they turn to non-parametric statistics, 3) they recognize that the data set of examples of fraud checks the accuracy of the classifiers, and therefore the evaluation of the data will be biased, since it will depend on how easily to establish fraud, having available data and unrecorded cases do not allow objectively evaluate the effectiveness of the classifier.
In this paper, we approve the following provisions: 1) We represent a potential hacker of the DM system. In previous works, the possibility of the existence of an “advanced” cracker was never taken into account. This threat model is especially important for digital counters, since an attacker who has access to an opened counter can send an arbitrarily selected fine-modular signal with an accuracy that was impossible with mechanical breaking (for example, using a strong magnet). 2) We offer a new system for measuring the accuracy of classification of abnormal indicators. This new metric system takes into account some of the main problems in detecting anomalous indicators in light of the safety issue: (a) the fact that the hacking cases recorded in the data set may not be characteristic of future hacks (therefore, a classifier able to fix these cases may do not recognize new more “advanced” hacking technologies), (b) in many cases it is rather difficult to obtain data on hacks for academic research - this is especially true for reducing s Integrated automated dispatch control system (CAS control unit) and the sensor data and actuator in the industrial and energy systems - that's why we say that we need to abandon the analysis and evaluation of classifiers with an unbalanced and irrelevant data set. (3) Using real-life IIM (data from fifteen-minute intervals collected over 6 months from 108 users) provided by the power company, we evaluate ways to detect electricity theft, including a new ARMA-GRL sensor created to capture the characteristic cases of illegal use of electricity (reducing electricity bills) in the formal model of a complex hypothesis.
Assessing Classifiers in a Hazardous Environment
In this section, we describe new common ways to evaluate the performance of classifiers in a hazardous environment. Since this concept can be used to solve other problems, we propose a model within the framework of a general classification. Our attention will be mainly focused on two topics: (1) the classification of hazardous conditions or how to evaluate the efficiency of the classifier in the case when a burglar performs an undetectable burglary, and (2) study dangerous situations or how to prevent the burglar from entering incorrect data.
Classification of hazardous conditions.
In self-learning machines, classifiers are traditionally evaluated on the basis of test data containing negative (normal) and positive indicators (tampering, hacking). However, in dangerous conditions, there are many situations where it is impossible to obtain in advance examples of cases of hacking. Two reasons can be distinguished here: (1) we cannot by definition get data of hacks on “zero days”, (2) using examples of hacks fixed separately from the classifier means that the hacker does not adapt to existing conditions and will not try to avoid our detection mechanism.
In this study, we argue that instead of using a number of hacking examples to evaluate the classifier's performance, we should determine the worst possible hacking option for each classifier and evaluate it, taking into account the financial damage caused by this hacking option.
Models and assumptions:
We will model the problem of evaluating classifiers, presenting the worst hacking options as follows:
1. A random process generates an observation x ∈ X. These observations are the realization of a random vector X with the distribution P0.
2. Suppose that x can only be detected using a touch sensor (that is, a smart counter), and the touch sensor sends y to the classifier. Thus, while everyone knows P0, a specific example of x is known only to the touch sensor.
3. The sensor can be in two states (1) normal and (2) abnormal. If the sensor is in normal condition, then y = x. If anomalous, then y = h (x), where h: X → X is such a function for which the permissible distribution P1 for Y satisfies the Attitude (cracker's intentions): g (X) R g (Y) (that is, E [Y ] <E [X] where E [X] denotes the expected variable X).
4. The classifier f: X → {n, p} outputs the solution: negative n when deducing that y is an example of P0 and positive p if y belongs to P1.
Metric system for evaluating classifiers in a hazardous environment:
To simulate hacking, we introduce the cost function C (xi, yi), which forms the vector yi, modifying the original value xi in such a way that yi denotes an example of hacking, which gives the maximum value C (xi, yi) if it remains undetected. Namely, we have the following:
1. The data set N = {x1, ..., xm} ∈ X m where each xi is considered an example from the set P0. Note that xi ∈ X. A common example is X = Rd, that is, every observation xi is a vector of real-life values with dimension d. When using a smart meter, this may mean that I xi refers to meter readings recorded in 24 hours.
2. The value α∈ [0, 1] representing the upper permissible limit of a false alarm is in the data set N.
3. The cost function C: X × X → R denotes the cost of damage from a false negative indicator.
4. A data set of one of the classifiers F = {f0, ..., fq}, where each classifier has parameters in the limit τ used for decision making. If we want to leave unknown the limit used by a separate classifier, we use the notation fi, τi.
This is only the beginning of the translated work.
The full text of the translated article can be downloaded here.
Literature:
1. EWMA Control Charts, itl.nist.gov/div898/handbook/pmc/section3/pmc324.html
2. forecast package for R, robjhyndman.com/software/forecast
3. RapidMiner, rapid-i.com
4. Antmann, P .: Reducing technical and non-technical losses in the power sector. Technical report, World Bank (July 2009)
5. Appel, A .: Security seals on voting machines: A case study. ACM Transactions on Security Systems 14, 1–29 (2011)
6. Bandim, C., Alves Jr., J., Pinto Jr., A., Souza, F., Loureiro, M., Magalhaes, C., Galvez-Durand, F .: Identification of energy theft and tampered meters using a central observer meter: a mathematical approach. In: 2003 IEEE PES Transmission and Distribution Conference and Exposition, vol. 1, pp. 163–168. IEEE (2003)
7. Breunig, M., Kriegel, H.-P., Ng, RT, Sander, J .: Lof: Identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, pp. 93-104. ACM (2000)
8. Brodsky, B., Darkhovsky, B. .: Non-Parametric Methods in Change-Point Problems. Kluwer Academic Publishers (1993)
9. Davis, M .: Smartgrid device security. adventures in a new medium (July 2009), www.blackhat.com/presentations/bh-usa-09/MDAVIS BHUSA09-Davis-AMI-SLIDES.pdf
10. De Buda, E .: System for accurately detecting electricity theft. US Patent Application 12/351978 (January 2010)
11. Depuru, S., Wang, L., Devabhaktuni, V .: Support theft. In: Power Systems Conference and Ex position (PSCE), 2011 IEEE / PES, pp. 1–8 (March 2011)
12. ECI Telecom. Fighting Electricity Theft with Advanced Metering Infrastructure (March 2011)
13. Geschickter, C .: The Emergence of Data Management (MDM): A Smart Grid Information Strategy Report. GTM Research (2010)
14. Krebs, B .: FBI: smart meter hacks likely to spread (April 2012), krebsonsecurity.com/2012/04/fbi-smart-meter-hacks-likely-to-spread
15. Lesser, A .: When it’s big IT goes on smart grid (March 2012), gigaom.com/cleantech/when-big-it-goes-after-big-data-on-the-smart-grid -2
16. McLaughlin, S., Podkuiko, D., McDaniel, P .: Energy Theft in the Advanced Environmental Infrastructure. In: Rome, E., Bloomfield, R. (eds.) CRITIS 2009. LNCS, vol. 6027, pp. 176–187. Springer, Heidelberg (2010)
17. McLaughlin, S., Podkuiko, D., Miadzvezhanka, S., Delozier, A., McDaniel, P .: Multi-vendor penetration testing. In: Pro-ceedings of the Annual Computer Security Applications Conference (ACSAC) (De-cember 2010)
18. Nagi, J., Yap, KS, Tiong, SK, Ahmed, SK, Mohamad, M .: Non-technical loss testing for power users using support vector machines. IEEE Transactions on Power Delivery Systems 25 (2), 1162–1171 (2010)
19. Nizar, A., Dong, Z .: Identification and customer of customer behavior irregularities. In: Power Systems Conference and Exposition (PSCE), pp. 1–10 (March 2009)
20. Peterson, D .: AppSecDC in review: Real-world backdoors on industrial devices (April 2012), www.digitalbond.com/2012/04/11/appsecdc-in-review
21. Smart Grid Interoperability Panel, editor. NISTIR 7628. Guidelines for Smart Grid Cyber Security. NIST (August 2010)
22. Sommer, R., Paxson, V .: Outside the world: On-line intrusion detection. In: IEEE Symposium on Security and Privacy (2010)
Source: https://habr.com/ru/post/213307/
All Articles