Modernizing Electric Utility Grids with IoT

Curt: We are seeing, with our customers, the whole electrification of the world, from our cars to our trucks, buses, homes, lawnmowers, even our leaf blowers.

To accommodate all this need, our transmission systems need to increase their capacity by 60% by the year 2030. And when we look at the grid today, 70% of the grid infrastructure is at least 25 years old. And utilities, when they invest in equipment, they expect these investments to last 30 years. So we've got a challenge here.

The government stepped up, and they've earmarked 65 billion of investment for grid modernization. This was in our $1.2 trillion infrastructure bill.

But our grid wasn't designed for distributed energy. It was designed for large, centralized generation. So, what we're seeing in our customer base, in the states that they play in, is that there needs to be a change.

And we see in places like California, which is very progressive in their policies towards renewable energy, they're creating different needs for the grid: solar systems and inverters, battery storage systems, and EVs. Today, all those sit behind the meter too or the business. And the grid was not designed to accept or manage power from these sources.

So, these DERs, or distributed energy resources, operate today autonomously. But they really need to become part of a manageable system.

In California, there's something called Rule 21 that started back in 2012. And it's being continued to iterate on a standard of the IEEE 1547. This is a way for these assets to connect to the grid to be monitored, managed, and controlled. So, we really need a system for integrating all these assets. And this drives us to investments in reliable, secure two-way communications to bring these assets into the system.

And this is driving our customers to invest in connected systems more than they ever had before. And what we're seeing is a lot of our large investor-owned utilities are either exploring, evaluating, testing, or deploying private cellular networks. And these investments are not small. They can be, you know, $100 to $400 million, so these are very big investments in upgrading the grid.

Curt: Well, I think one of the things we're seeing is a lot of investments in things that make the networks more secure. We've been providing cellular solutions on public networks and then also on unlicensed frequency for things like Zigbee and smart home and smart energy for 20 years.

And when I look at the cellular side, I'm not aware today of any attacks that have happened through those networks with any of our customers, which is interesting. But we see a lot of attention and focus by our customers as they're looking at their networks, and we're seeing things like ransomware and cybersecurity being integrated into the IT technologies that have been used for years in the operational networks that they run their business on.

And as we look at things like private cellular networks, that's one of the areas where they're taking it off of the public network and making it a totally private, non-shared network for managing their grid.

Ben: One of the other things that we're seeing is that as communication flows become more and more critical that people are looking for ways to implement application-specific data flow controls. And some of the equipment that we're providing gives the ability to programmatically add that sort of capability into the data stream so that you have essentially data diodes that prevent data flows, when they're not supposed to be encouraging data, to remain segregated from each other and thus preventing access to equipment when it's not needed or allowing the data flow only in one direction.

Alec: I think it's important too that all these technologies maintain and fit the NERC CIP and security protocols that are out there, such as a lot of the kind of ransomware attacks and things that we're seeing today are actually generally attributed to social engineering and someone gaining access to particular equipment locally and being able to input flash drives and other things that can then start to attack the system. So it's important to make sure that all of this information, to Ben's point, is segregated and housed in DMZs. And it's able to get in and out of the system, but in a secure way, where it can't attack the whole system at once.

Alec: Asset monitoring actually can be a key aspect in this. As we've already talked about, there are lots of new sources of energy that are entering into the grid now. And it's important to monitor how much of that energy is available and how to get it where it needs to go. So it’s not just having the transmission capabilities but it’s also important to understand where that energy is being generated and getting it to the right location. So load management can be a huge aspect with this today that I think can be tackled directly using products that can monitor your assets.

Curt: We were just at DISTRIBUTECH 2023 as an exhibitor. And there was an awful lot of attention this year, more than I've seen before, in monitoring of the system, putting sensors out, particularly in the medium voltage world where they're putting a lot of sensors out on the lines and doing instrumentation, and have systems that are doing that monitoring proactively on a real-time basis.

I see a lot of investments of instrument technologies sitting on top of the grid, watching what's happening. And to Alec's point, the whole monitoring of the new things in the distributed world that are feeding in are critical. And I think that kind of ties to what we're seeing with Rule 21 and those sorts of things in California.

Ben: So, this is really a large challenge for all utilities in the generation, transmission, and distribution space today. To Curt's points earlier, there are many different laws being put into place, and these are changing regularly.

So as the tools that allow for this as well as the standards for communication and the laws are all changing, I think it's important that you can rely on a company, such as Digi that can provide equipment to support these communication aspects in a secure way.

Particularly in the electric vehicle space, there's a new standard called GreenPHY that is being used to connect and communicate between everything from the car itself transmitting data, all the way up to the actual grid. And all of these elements are having to work in tandem to resolve these issues and provide valuable charging assets for these EV vehicles that are on the road. This is true both in the commercial EV charger space as well as the home automation space. So it's important to have certain standards there.

Another asset that's really undergoing broad changes today and needs updated technology to address is the solar panel space. This is again an area where it spans all the way from full commercial, industrial, solar, and wind power firms,  to individual consumer homes and how are we going to monitor that transfer of power and data in this environment.

With all these challenges, it's more important than ever that utilities have a smart, connected network that can gather and standardize this information into actionable items in your own SCADA system or your own operations. So this network connection must be very reliable in order to meet the mission-critical needs of that utility space. And generally, these can be connected to the broader ecosystem via wired connections and/or general RF products.

So some of the technologies we've talked about today including cellular, including public and private, as well as 900 megahertz and 2.4 gigahertz technologies like Bluetooth, Matter, Wi-SUN, and Wi-Fi. In order to cover this broad spectrum, it's important to integrate these technologies into your product in order to be able to gather all of that data and transfer it in a secure way.

And with that, it's also important that the availability of that information is at your network operator's hands. Which means it's important to integrate all of this equipment so you can focus on your key objective, which is having a reliable network for your customers. Digi provides a lot of products that can create an intelligent system to be able to handle all of these and be the subject matter expert on all of these items, such as protocols and wireless technologies.

Curt: So today, we see cellular networks used for management and monitoring of typical distribution assets like cap banks, reclosers, and substations. We also have a number of applications in commercial and industrial. These include metering, power quality management, and demand response and load shedding. As Alec mentioned, there’s also the connection of all the new renewable sources, whether commercial or residential.

And cellular is also used for backhaul of AMI networks, which today are typically mesh networks operating on unlicensed frequencies, like 900 megahertz or 2.4 here in the U.S. We believe as utilities deploy private cellular networks, there's a possibility that some of these AMI networks may be replaced by cellular. However, there's pretty significant investments in those current AMI networks, and they're probably going to be around for a while.

Alec: To Curt's point, we're seeing a lot of distribution utility companies using our equipment today in order to handle load shedding. As you know, plugging into the current connected device's ecosystem can be a very large challenge. Implementing cellular and a load-shedding smart switch that customers can opt into is a very viable solution for quickly managing network capabilities and making sure that you don't have too much load on the network.

Curt: First, let's start with public networks, which our customers have been using for the last 20 years. One of the benefits, obviously, is they're fairly ubiquitous and available to enterprises without any capital investments.

And one of the things that we've seen over the last 20 years is, as we buy more and more smartphones, those networks get bigger and bigger and faster and more reliable. So it's great that our enterprise customers, like utilities, can leverage that investment.

But they are public, so they're shared. And during peak traffic loads, let's say late in the afternoon in a metropolitan area where everybody's driving home and they're all on their phones, service can degrade. So there's some unpredictability in those networks.

There's also the perception that public networks are less secure, or they're perceived to be. As I shared with you, our experience in the last 20 years is we really haven't seen any security issues on public cellular networks with our utility customers. But on a private network, the customer does have control over the SIM, which provides the access to the network. So there is definitely some additional security in being able to turn that SIM on and off.

Private networks do require huge investments. I talked about putting up a private network can be in the hundreds of millions of dollars. This is for acquiring licenses for frequency, which are very expensive, and building out the infrastructure. And then probably one of the most often overlooked and significant investments is the ongoing operating cost of running a private network.

So, in essence, if you put up your own private network, you become a mobile operator. And just like running an electric grid, you have responsibility of running, monitoring, and managing that on a 24 by 7 basis, for 365 days a year, so lots of cost and responsibility.

Private networks, though, will provide reliable, predictable and deterministic performance, which is nice for things like controlling the grid. So when you don't have anybody else on your network and it's just your assets, you're going to have higher levels of performance, throughput, lower latency, and, more importantly, fewer drops and retries on communication sessions. With private networks, you also have the ability to control different levels of service for different applications. So, you could take something like communications to a substation and give that much higher priority over a meter read at a residential network.

But another thing is there are hybrid approaches to cellular networks. It doesn't have to be all public and all private. You can actually use a combination of both. Most of our customers will because they'll never build out complete coverage.

But the public carriers also have things like AT&T FirstNet, which a lot of our investor-owned utility utilities use today, which is a public network that's built for very specific mission-critical applications so that your applications are not contending with all the public traffic on a network. So it's a kind of a hybrid approach and gives you a little best of both worlds — great performance and security — but on a shared asset that you don't need to invest in.

As I shared, all of our customers are looking at building out private networks. And if they do build out a private network, they will still use public networks, likely for backup and resiliency, because networks do fail. And they would use things like dual SIMs in their devices or SIMs that are programmed to failover from a private network to a public network and back on the private network when the private network recovers.

Alec: I haven't heard of a particular customer instance or anywhere where this is being regulated to that extent today. I know that this is something that's been going around as a way to potentially offer better services to customers. But I think that it would create a whole restructuring of the grid and something that's a very large challenge for IoT to deal with.

Alec: So, basically the determination you're making there is kind of a wide area network, such as the cellular networks versus a local area network. And Digi does not offer Sigfox solutions, but we do offer some very low-power LP WAN dedicated solutions.

We have solutions like LoRaWAN connectivity within our OEM solutions business unit as well as other solutions where you can create your own local network, such as our XBee solutions, and XBee Intelligent Edge Controller, and then connect out to a gateway that can be located in a separate location that can provide that wide area network connectivity to those internal assets.

Curt: Just to add, I think Sigfox has kind of gone out of business here in North America. I think they're still operational in Europe, probably France, but that business model was not very successful.

To Alec's point, we do offer some solutions using like licensed frequencies, and they would fit in the area of 902.4 gigahertz and using our technologies like a standard like LoRaWAN and then a proprietary offering we have called DigiMesh.

Curt: That's a really good question. I look at a lot of what we're involved in. And I would say it's more monitoring and management than it is control. Exception would be communications to things like substations. But, you know, I think a lot of what we do is involved in monitoring.

Command, obviously, that's a control function. Alec talked about that earlier, where customers, whether they're residential or commercial, can opt-in to a demand response program where they're offering control of their usage to the electric utility.

Alec: Yeah, I would echo exactly what you said there, Curt. From what we see on the OEM side, it’s probably 75% to 80% monitoring and diagnostic today. Generally, it is a predictive-maintenance-type activity that is going on and monitoring of sensors and other items within the grid in order to determine when maintenance is needed.

We do see, as one of the scenarios I gave there, that there are some control aspects that are being implemented. And I do believe that many of the customers that are implementing monitoring, simply haven't implemented the actual technologies to do the controls at the device level there as well. But this is something that could potentially be added via these communications in the future. So, I'd say that we're not prohibiting anybody from entering that space via these technologies, it's more of just the equipment that's needed to do so today.

Ben: The initial rollout of 5G focused heavily, in the press, on the high bandwidth, low latency kinds of applications, such as video and communication for autonomous vehicles. The reality is that really 5G is set up to offer a differentiated range of solutions across multiple different frequency bands and with greater ability to segment data based on priorities. I think that'll tie into the question that was just asked about monitoring and control versus monitoring.

While current 5G deployments are still concentrated on serving dense population areas and the higher bandwidth applications, the future radio chipsets are going to start coming out. We're starting to see samples of them now. They'll probably be on the market in the next year to two years, and will offer cost-effective connectivity for a lot of these different critical infrastructures, as well as the ability to use the lower frequencies to get better penetration and range, and to differentiate things to allow different kinds of communication to be segmented from each other. It's going to give a lot of flexibility to implement these kinds of capabilities.

Digi supports a technology we call our CORE module, which is a plug-in modem that allows for flexible deployment of whatever the current available technology is at a given location, as the infrastructure supported. And so, as we've said before, a lot of these implementations are designed to be put out in the field for 10, 20 years.

And obviously, cell technology changes quickly. So this allows for updates to cell technology without making profound changes to how the communication works, which again is going to tie back to some of the security questions that we asked earlier. So this same technology allows an easy switch between pure public communication or CBRS private cellular networks.

The FirstNet or the existing 5G or the new 5G technology is going to be implemented with the simple change of the radio without having to reconfigure how the device is communicating.

Curt: I'd like to add to that. Most of our customers that started in the world of 2G and then 3G, really even didn't need 4G for a lot of their applications. The speeds and the latencies were more than adequate for grid applications. Typically, we're pulling meter data or state data.

And sometimes, as we talked about, there’s a simple command to change the state, and those don't require very much bandwidth. The exception is assets like substations, where you have a lot more equipment and responsibility for downstream transmission. They could benefit from higher speeds and lower latencies.

What's interesting to me is we have a number of customers that put our equipment out and they’ve run it for 10, 13,14 years and never upgraded. Utilities don't like to make changes to things that work, but they were forced to upgrade by the mobile operators, turning off their 2G or 3G networks and causing them to replace their equipment.

We had a discussion earlier about the benefits of public versus private networks. One of the nice things about a private network is you determine things that need to change. And if they don't need to change, you can keep them the same. And you're not forced by a mobile operator. So, I think most of the stuff that we've been involved with, for the most part, 5G hasn't been required.

Alec: Particularly in the IoT space and in utilities, no, we haven't really seen that need for the 5G speeds. But I think where the need is going to come is exactly where you were talking about. Typically, these cellular generations last about 10 years, depending upon the location, right?

And we're seeing now that 2G and 3G has been basically entirely phased out of the tower technologies. Eventually, 4G LTE will go that way as well. But the nice aspect of LTE, which stands for Long-Term Evolution, is they actually had a plan for this with IoT devices, and the 3GPP spec has already been included for 5G to include LTE networks, low-speed IoT networks, such as LTM and NB-IoT, into the future.

So I think we'll still see the continuation of these IoT networks as the generations move forward and will still need these low-power, low-bandwidth devices that are out there.

Curt: I think it could be either. I probably have not seen a lot of satellite use within electric utilities. We've seen satellite use in a lot of other verticals that we play in, particularly, oil and gas, things like mining, forestry, maritime, but not a lot yet in the electric grid.

Alec: I would agree with that. Typically, satellite is going to be constrained given the latency that comes along with that and the potential cost of actually putting the infrastructure up. Although that is being addressed somewhat today via some of the companies that are out there.

But I'd say that, generally, we do see those low power networks using devices like our XBee to transmit information back to a gateway. Particularly, we've seen this be very successful in solar and wind farms, where they're in very remote areas, and they can all transfer information back to a single gateway that does have that cellular connectivity available.

Ben: We've started to look at time-sensitive networking. The integration of time-sensitive networking with cellular is something that's going to become more viable as we move towards 5G segmentations. The biggest issue is having a continuous path to prioritize the data.

And so, yes, we are looking at those technologies. For these kinds of applications, that's not the core functionality that we're seeing, but it's stuff that we're pursuing and monitoring.

Curt: I think for those types of applications, we do offer a number of products. To Ben's point, 5G offers the very high speed and the low latency. We also know that networks today, a lot of the wired networks, are less reliable. So where somebody isn't running to fiber to a substation or an asset and using wireline connections, wireless is probably going to be more reliable and a lot more cost-effective.

And we do have products today that support 5G and support 5G with dual radios. So we do offer a level of resiliency where you can use two connections, say one on a public network and one on a private network, or both on different public networks. And we also offer a bond thing of traffic over both those connections to increase the throughput and lower the latency. So there are some solutions for those types of applications.

Curt: If you look at our products on our website, you'll see products like Digi TX54 and TX64 5G that offer dual cellular. And all of our cellular products — as I said we're a pioneer on this — offer dual SIMs per cellular radio. We have a technology we introduced a long time ago called Digi SureLink® that was copied by a lot of others that offered that ability to failover between one network to another network or one SIM to another SIM.

And in the case where we have dual cellular radios and a device, we have two SIMs for radios, so you'd have up to four SIMs. So that would give you the ability to failover between four different networks, whether they be public or private.

So, you get a lot of resiliency and agility. Of course, there's a cost associated with all of it. It's a more expensive piece of equipment. And if you're using public networks, you're paying for multiple service plans. If you're on a private network, you don't have any recurring costs. You're the operator of the network. And so that would offer some savings.

And in fact, that's another benefit that I didn't mention of private networks is the elimination of service fees. In fact, that's one of the justifications, in the rare cases, that the utilities are doing for investments in private cellular, the elimination of recurring fees today that they pay to mobile operators.

Alec: I think it's been mentioned here a few times, but a lot of this is going to come from regulation. And it's going to be states making sure that all of the equipment is interoperable and can intercommunicate between devices, right?

The company that's creating the actual solar panels, and the company that's actually running that solar panel grid and introducing it back into the smart grid — today, these are separate entities. And with standards and interoperability regulations that are making those requirements, it’s likely going to be the thing that's going to harmonize this and make it so everyone has to work together versus everyone struggling to get a certain percentage of the market.

Curt: Yeah, and I think another thing that we're seeing — and we have some customers in this space and we're also exploring partnerships in this space — is the companies developing the software that didn't exist in the past that does the aggregation of the communications from all these devices back into the grid and then provides the communications back to these devices to do the control functions that are needed.

So there's an emerging set of companies that are making end software that may sit on a Digi device or be embedded into that inverter. And then there's aggregation software. A lot of it today is cloud-based, and some of it is enterprise-based, and a lot of the companies offer both. They do that aggregation, and then integration through APIs into the control system of the utility.

Curt: Most of the stuff that we've been working with the new distributed energy sources, whether they're microgrids, solar farms, or things at people's homes, play in those large, centralized generation facilities. There's a lot of communications infrastructure going to those assets that are primarily on the utility's fiber network. So, typically, Digi hasn't been involved in those types of applications.

Ben: That also plays into the thing that Curt was just talking about. There's a lot of equipment out there, whether it's baseload or more classic peaking plants, or intermediate substation and distribution line monitoring equipment. Some of that stuff has a network stack in it, and a lot of it really still has serial ports because they work.

And one of the big issues that you have is that those embedded network stacks tend not to be something that's easy to update in the field. If you're using mission-critical equipment there's a lot of testing and validation that has to go in, and it's critical equipment. Taking it offline for maintenance is a risk and a time effort.

Whereas using the gateways and connectivity solutions that we offer allows you to delegate the communication function to something that's easier to maintain and manage. And Digi has taken a very proactive stance to keeping current issues addressed to providing centralized management functionality that allows it to be very easy to update, even with thousands of units in the field, without affecting how the grid is operating.

So the ability to make these connections to all these critical assets and to maintain connectivity for the current application solutions that are in place means that you can increase the reliability of access to the information, increase the flexibility of how you get there, bring those connections into a private network, and move them to 5G when needed, rather than having to constantly go and update critical essential equipment.

Because of the nature of what our products do, we're able to make sure that we can afford the decades-long deployment that these products are in play, maintain connectivity to all of this critical infrastructure equipment, integrate with the new equipment that's starting to be placed out there, help address NERC CIP requirements (security, best practices), keep communication paths secure and segregated, put edge intelligence out in the field where it can help monitor, alert, alarm, react, and integrate different generations of equipment, and make that all seamless to the operating authority.

Curt: We've added into our products recently, we've added a lot of edge computing capability, particularly in our operating system. We now have something called Containers. So we support Linux containers.

One of the things you can do with edge computing is you can monitor state data, for instance, at the edge and send only when things have changed or when things are out of bounds. So, what it allows you to do is to continue to operate if a connection goes down, but also limit the amount of traffic that you're sending up over the network by adding intelligence out on the edge.

Alec: That's a good answer. And it really does get included in all of our products. Even our XBee products have edge intelligence capabilities via MicroPython on board in order to be able to interrogate that data and only send when necessary.

The other thing to note here is it's important to define kinds of security measures that need to be taken on certain data and potentially other data that's not because there's always tradeoffs, right? You can send data via a very short message UDP protocol over cellular, you know, data that doesn't necessarily need secure levels to keep that data away from others. And then there's also data that you need to keep very secure. You need TCP socket connections and VPN connections, right?

So it's important to define where that data falls in those buckets, and the Digi devices can all help you send that data through those different channels.

Ben: To follow on to that, we also have professional services and implementation engineering groups that can help work with the operating authorities and the SCADA system integrators to help make sure that best practices are being followed given the business objectives of the system that is being implemented.

Alec: The biggest challenge is today that we're being faced with that because the standards are just being defined. It's the interoperability challenge. There are a lot of different standards out there, everything from Wi-SUN, Matter, LoRaWAN, Sigfox, Wi-Fi, BLE, and cellular, right? These are all different technologies and protocols that can be used to transmit information.

I think it's important to note the large majority of those that I just mentioned that exist today. You could implement Digi products, and you don't have to be the subject matter expert. You can just deploy whichever protocol and product fits the scenario that you need to operate with, and then you can focus on your own expertise and actually what data is important to you.

Curt: My answer would be people probably are the biggest challenge. And as I look at a number of the customers that we've worked with over the years, a lot of the people that we started with early in the IoT journey, a lot of them are retiring. You know, I think everybody knows, we have a shortage of workers in the technology world. It's getting harder and harder to recruit people.

As I look at what's going on the migration from public to private networks where utilities are putting up their own networks and becoming mobile operators, they're very, very reliant on consultants and outside parties to help them do that, while going through an RFP stage, going through pilots, doing the engineering, doing the deployments, and then running them. They're very reliant on outside parties that have people and talent to do that. So, the biggest challenge is people, and training people and hiring people and reliance on that human capital.

Ben: I would tend to agree with the assessments that both of my colleagues made. I tend to phrase it as keeping current on technology. The rate of change in technology keeps increasing, and just keeping track of what the different options are and what makes the most sense.

And so, in that situation, it really makes the best sense is to have a partner who is familiar with all the technologies, from the oldest and most mature ways of doing things to the newest and most potentially interesting, somebody who understands how connectivity needs to work across a wide variety of different options, and who can understand how that has to fit into the complex infrastructural systems that represent the grids we have and especially the grids that we're building for the future.

And so in that context, that's who Digi is. We've been doing this kind of communication across a wide variety of networks for all kinds of critical infrastructure for decades. And that's why we are trying to reach out to make sure that we can offer to help people address these solutions as they're trying to modernize their grids to meet the demands of tomorrow.