Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
This chapter gives an overview of the principles used defining and implementing the architecture. The following principles were applied:
Layering
Domain driven design
Dependency inversion principle
Behavior driven development
The use of layers improves the separation of responsibilities. Each application contains the following layers:
Presentation layer: responsible for providing information to users (persons and/or systems) and the handling of user requests
Application layer: responsible for executing system tasks including authorisation control
Domain layer: responsible for the representation of the problem domain.
Image, Layers:
Audit logger
Web Services
Functions
Domain-driven design focuses on the problem domain. DDD's starting point is creating an optimal model for a specific problem domain by having a common language and constructive collaboration between technical and domain experts.
DDD uses the following building blocks:
Entity: An object not identified by its attributes but by its own identity.
Value Object: an object with attributes but has no own identity.
A collection of objects surrounding a specific root entity (or aggregate root). To ensure consistency objects in the aggregate can only be addressed through the aggregate root.
The dependency inversion principle promotes an independent connection by inverting dependency relations. This ensures that the domain model can be very 'clean' without knowledge of the underlying infrastructure (POJO classes). The Spring framework is used to implement the Dependency Inversion principle.
Behavior driven development is a way of programming that first describes behavior in user stories and then implements this in code. The user stories contain scenarios with acceptation criteria that can be automated. This creates a complete test suite for the whole system.
For the application of BDD the following frameworks are used:
Cucumber and Gherkin, automated acceptance testing, based on scenarios from stories.
Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc
Workflow engine
Protocol framework
Protocol implementations
Workflow engine
Queue
Communication
Service: Contains instructions not related to a specific object.
Repository: Serves as a collection for fetching and saving objects. Creates an abstraction for actual persistent implementations.
Factory: Contains methods to create domain objects.
Image, functional layers overview
The Functional view shows an overview of the most important functions of the system. The two images below show the starting architecture and functional reference architecture respectively.
Image, functional starting architecture
Web applications
Open Smart Grid Platform
Web services
This model partitions the system in seven functional clusters (vertically) which are shown on the system layers (horizontally). The circled numbers refer to image 1.
Image, functional reference architecture
Vertical clusters:
Device installation
Device management
Firmware management
Horizontal System layers:
Web applications
HTTPS/SOAP communication
Platform
Time behavior is mainly important in the Flexovl application when a lot of devices have to be addressed in a short period of time over a wireless network. Both latency and limited bandwidth have to be taken into consideration while demanding the coordinated on and off switching of the lighting, since we want to avoid the Christmas tree effect.
Time synchronization: devices periodically register with the platform and receive a time.
Protocol: because of the limited bandwidth an efficient protocol "protobuf" was selected.
Light metering messages
When the SSLD's are disabled the PSLDs cannot be addressed
Because of these points of interest we use message queueing combined with a retry mechanism of delayed delivery.
The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.
Database
Communication infrastructure (CDMA/GPRS/Ethernet)
IP infrastructure
Open Street Light Protocol (OSLP)
Public Street Lighting Device (PSLD) or Sub Station Lighting Device (SSLD)
Schedule management
Ad-hoc control and status
Monitoring
Smart devices
This are some examples how a message flows in the Open Smart Grid Platform.
Step
Description
*In case the response is not timely retrieved by the client app, OSGP will resend the notification with correlation id to the client app. The amount of retries is configurable.
WS adapter retrieves (and deletes) response from DB
14
WS adapter sends soap response to the client app
1
WS adapter receives client soap request with organization certificate and organization id in soap header
2
WS adapter authenticates organization, checks authorizations and sends request message to domain adapter (via queue)
3
WS adapter returns soap acknowledgement with correlation id
4
Domain adapter sends request message to core (via queue)
5
Core determines protocol for device and sends request message to protocol adapter (via queue)
6
Protocol adapter translates domain request message, sends request to device and receives response from device
7
Protocol adapter sends response message to core (via queue)
8
Core forwards response message to domain adapter (via queue)
9
Domain adapter forwards response message to response queue
10
Client app polls for response using correlation id (with organization certificate and organization id in soap header)
11
WS adapter retrieves response message from response queue
12
WS adapter sends soap response to client app
Step
Description
1-8
Same as request/acknowledge/poll message flow
9
Domain adapter forwards response message to WS adapter
10
WS adapter stores response in DB
11
WS adapter sends soap notification with correlation id to the client app*
12
Client app sends soap request with correlation id to retrieve the response
13
The Open Smart Grid Platform contains an extensive authorization model, which enables a device owner to give certain rights on certain devices to other organizations. Every organization will only see devices they have rights to.
This model displays the most important entities of the open smart grid platform system and their mutual relationships.
Image of Logical Authorisation Data Model
The logic of the model above:
At the top of the image is the entity "authorisation". This represents the permissions of an organization on a certain device. In general an organisation will have a lot of permissions, at least one for each device it needs to manage.
The functions an organisation can execute on a device are determined by the function group the authorisation refers to. Function groups are collections of functions and are predefined in the software. The following function groups have been predefined.
Owner-group (this contains all functions)
Ad hoc-group (Functions for ad hoc switching of lighting)
Management-group (Platform functions)
This structure provides maximum flexibility when assigning rights to devices. Devices always belong to an Owner. An owner is an organisation, but not every organisation is an owner. A device can have more than one owner. The entity "Event", at the bottom of the image, is the execution of a function by an organisation on a device.
Details like device-type, device-status, etc. have been omitted from this model.
One security requirement is that each event must be traced back to a 'natural' person, also known as an audit trail. Although the open smart grid platform does not register individual users we can meet this requirement by registering a data-item with each event. This enables the user organisation to investigate which events belongs to which 'natural' person. This data-item can for example be an user-ID provided by the user organisation which doesn't have to be unique in the open smart grid platform.
Table describing the entities in the logical data-model
An organization can get rights to one or more function groups, and thus all functions in that function group will be available to this organization.
To ensure that devices can only receive instructions from a 'genuine' open smart grid platform it must be possible to authenticate the open smart grid platform. This is implemented through a standard technology based on asymmetric encryption (if supported by the Device). The open smart grid platform will receive a unique key to enable the devices to tell if the messages come from a 'genuine' open smart grid platform. Both OSLP and DLMS device types use this kind of encryption. To prevent replay-attacks each message will get an index number (this is standard practice as well).
To ensure that the open smart grid platform can distinguish between 'genuine' devices and 'illegal' devices, all devices are supplied with a manufacturer key. Each device has a unique key. Because of the asymmetrical encryption the platform contains the public part of each key. In this way devices can be identified by their unique key and their unique hardware ID. The device-ID will be encrypted in each message sent from the device to the platform.
All communication between the open smart grid platform and the devices will be signed with these keys to ensure (1) the source is legitimate and (2) to ensure the integrity of the message. It is not necessary to encrypt the whole message because confidentiality is not important. This results in a less computationally intensive process.
When a key is stolen (by hacking a device) this will not affect the integrity of the other devices. Each device has an unique key after all and only the hacked device has to be excluded from communication in the platform.
The security is independent from the carrier (GPRS, CDMA, Ethernet, etc.). The open smart grid platform supports symmetric and asymmetric encryption (depends on device and protocol).
For OSLP devices, the firmware will be used to distribute keys to devices. In this way we can use the existing secure firmware update mechanism for updating keys and certificates. DLMS devices use a mechanism to switch keys that is not dependent on firmware updates.
Additional security may be provided by using TLS communication.
Authorisation for use of the platform functionalities is handled by function groups. Function groups are defined for both platform functionality and device functionality. Each function group has one or more functions. Access to device functions can be set per device. The tables below show an overview of all function-groups and device-functions and platform-groups and platform-functions respectively.
The following security measures can be used in a hosted environment:
DDOS protection
IPSEC VPN connections
IP whitelisting
Most cloud environments support these features.
Hardened operating systems (according to Center of Internet Security)
Communication over TLS
Firewalls between all servers and layers
Certificates from a recognized Certificate Authority (CA)
For every major release there will be a mandated security test initiated by Alliander.
In cooperation with the European Network of Cyber Security (ENCS) state of the art security measures were implemented.
Security per device
Security per application
Security Certificates per Organisation and per device
Security measures: 1. Firewall in defined zone 2. Operating System Hardening 3. DDOS protection 4. Replay attack prevention 5. Private encryption key per device 6. Certificates from a Certificate Authority 7. Encryption via Elliptic Curve DSA 8. IPSEC VPN for CDMA and GPRS 9. Unique device identification 10. Unique CDMA modem number 11. Role based authorizations on functions and devices
An analysis of safety aspects has led to the decision that the safety of the whole system will be realized by proven technology based on asymmetrical coding (also known as public-key encryption).
Two-way SSL will be used between web applications and the Open Smart Grid Platform to verify the identities for both client and server. User organisations are responsible for the administration of the identity of and access to their web applications. The web applications feature a login page. After successful login the user is linked to an organisation. Passwords will be stored with encryption. The organisation ID will be sent in each message to the Open Smart Grid Platform and will be verified by the SSL certificate.
Only public encryption Algorithms will be used. Due to performance limitations (of the devices) and recommendations from The European Network for Cyber Security (ENCS) Elliptic Curve DSA with 256-bit-keys was selected. This improves the security and efficiency over the 1024 bit RSA algorithm. Messages can be smaller and less processor capacity is needed. The key length of Elliptic Curve DSA is similar to the 3072 bit key length of RSA.
Note: Even though the open smart grid platform uses ECDSA to secure the OSLP, other encryptions may be used as well. The RSA Algorithm is still supported if preferred. This is a flexible configuration option.
A private APN is used for linking to mobile data communication infrastructures.
Every action to and from devices is logged in the audit trail
Messages from unknown devices will be denied (and logged)
The Open Smart Grid Platform is designed and built for scalability and reliability
Messages will never get lost, In the worst case scenario, a message will be sent to the dead letter queue.
Any layer of the platform can be independently scaled up- and down
Adding servers can be done runtime
It can run in an active-active setup over multiple servers and data centers. In our cloud hosted setup even over sets of data centers in different countries.
Firmware-group (Functions for updating firmware)
Schedule-group (Functions to create lighting schedules)
Tariff scheme-group (Functions to create tariff groups)
Configuration-group (Functions to configure devices)
Monitoring-group (Functions to monitor devices)
A party playing a role in the management and control of the devices, for example municipalities.
owner
An organization role. Each device has an owner.
event
An event reported by the device (which is not the action of a natural person) for example an error or security problem.
FIRMWARE
SCHEDULING
TARIFF_SCHEDULING
CONFIGURATION
MONITORING
GET_DEVICE_AUTHORISATION
X
X
X
X
X
X
X
X
X
SET_DEVICE_AUTHORISATION
X
START_SELF_TEST
X
X
STOP_SELF_TEST
X
X
SET_LIGHT
X
X
GET_STATUS
X
X
RESUME_SCHEDULE
X
X
SET_REBOOT
X
X
SET_TRANSITION
X
X
SET_EVENT_NOTIFICATIONS
X
X
GET_EVENT_NOTIFICATIONS
X
X
REMOVE_DEVICE
X
X
UPDATE_FIRMWARE
X
X
GET_FIRMWARE_VERSION
X
X
SET_SCHEDULE
X
X
SET_TARIFF_SCHEDULE
X
X
SET_CONFIGURATION
X
x
GET_CONFIGURATION
X
X
GET_ACTUAL_POWER_USAGE
X
X
GET_POWER_USAGE_HISTORY
X
X
GET_MESSAGES
X
FIND_DEVICES
X
SET_OWNER
X
Entity
Description
authorization
Authorization – Permissions of an organisation to execute a certain function (member of a role) on a certain device
device
Electronics present in a "container" (for example a lamp post), connecting to the open smart grid platform and (in case of a lamp post) controls the lights. One device has one owner.
audit trail
The actions of an organization on a device. A combination of [time, organization, function and device].
function
An end-to-end operation. For example "set schedule". A function belongs to multiple function groups.
function group
Usually this will be larger groups than the "function clusters" in this document. All functions available to end users could be in a single group for example.
Groups
Functions
OWNER
INSTALLATION
AD_HOC
Groups
Functions
ADMIN
USER
CREATE_ORGANISATION
X
GET_ORGANISATIONS
X
X
GET_DEVICE_NO_OWNER
organization
MANAGEMENT
X
Role based authorizations on specific functions of devices
Access control
Unique device identification
This chapter describes the results of a performance test, to give potential users an indication of the system requirements for the platform.
The Platform was tested with the following AWS setup:
Systems:
Specifications Component Server: 2 CPU's, 8 GB RAM
Specifications Database Server: 1 CPU, 2 GB RAM
Setup:
Front-end: 2x Component Server
Middle-end: 2x Component Server
Back-end: 2x Component Server
For the test to succeed, two requirements were to be met:
Switch 10.000 simulated OSLP devices under 5 minutes.
Switch 40.000 simulated OSLP devices under 5 minutes.
The results showed that both tests succeeded.
The Core layer of the Open Source Grid Platform is responsible for Validation, Translation, Authorisation and Routing of request messages. It also contains all the Domain Objects.
The core layer consists of two components:
osgp-domain-core: Shared Domain objects, services, repositories, etc. These classes are used through the entire platform.
osgp-core: Logic for routing domain requests, scheduling, retrying, etc.
entities: Defines the entities used for persistence.
exceptions: Domain specific exceptions reside here.
repositories: Repositories that contain logic for persisting entities.
config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.
-- ApplicationContext
-- OsgpCoreInitializer
-- DomainMessagingConfig
-- PersistenceConfig
-- ProtocolMessagingConfig
These packages contain interfaces for the Services.
domain: Interfaces for the Domain services.
protocol: Interfaces for the Protocol services.
domain: Contains Messages, MessageListeners and MessageProcessors for Domain related messaging.
protocol: Contains Messages, MessageListeners and MessageProcessors for Protocol related messaging.
The Domain Adapters are responsible for receiving requests from the Web Services layer, and delivering them to the Core layer. The Domain Layer mainly contains MessageProcessors and Services for request handling.
The Core/Admin components contains the shared functionality, while the Domain components contain additional domain specific functionality.
At the moment the Platform uses the following Domain Adapters:
Generic
Core - osgp-adapter-domain-core: Contains Core (common) functionality; AdHocManagement, FirmwareManagement, etc.
Admin - osgp-adapter-domain-admin: Contains Admin functionality, e.g. DeviceManagement.
Domain
Public Lighting - osgp-adapter-domain-publiclighting: Contains functionality for the Public Lighting Domain.
Smart Metering - osgp-adapter-domain-smartmetering: Contains functionality for the Smart Metering Domain.
Tariff Switching - osgp-adapter-domain-tariffswitching: Contains functionality for the Tariff Switching Domain.
config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.
-- ApplicationContext
-- DomainAdapterInitializer
-- MessagingConfig
-- PersistenceConfig
jms.core: Inbound/outbound messages from/to the Core layer.
This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Core Queue, or receiving and processing responses from Core.
jms.ws: Inbound/outbound messages from/to the web services layer.
This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Web Services Queue, or receiving and processing responses from the web services layer.
ActiveMQ Front-Middle: 1x Component Server
ActiveMQ Middle-Back: 1x Component Server
Databases: 1x DB Server
specifications: Interfaces that define specifications for Devices and Events.
validations: Validators and constraints.
valueobjects: Definitions of the Domain Objects.
services: Services that process device requests/ responses. Checks for authorization, and if the request is supported by the platform, it will be routed to the appropriate protocol adapter.
tasks: Contains task scheduler logic.
Microgrids - osgp-adapter-domain-microgrids: Contains functionality for the Micro Grids domain.
Distribution Automation - osgp-adapter-domain-distributionautomation: Contains functionality for the Distribution Automation domain.
services: Contains most of the domain logic, related to the specific services of the adapter. The service classes converts DTO objects to Domain objects (or vice versa), and put the request on the Core queue through the JMS classes.
The Web Services layer contains the web services that are used to communicate with the Platform. The Open Source Smart Grid Platform uses the Simple Object Access Protocol or SOAP to expose its interfaces. The Web Services Adapter receives requests and sends those to the Domain Adapter. An incoming request is converted to a Domain Object, and put on the MessageQueue of the Domain layer. The Web Services layer also has a queue for incoming responses from the Domain adapter.
Each domain of the Platform has its own web services:
Core - osgp-adapter-ws-core: Contains the Core (common) web services.
Admin - osgp-adapter-ws-admin: Contains the Admin web services.
Shared - osgp-adapter-ws-shared: Contains shared endpoints, such as header authorization
Database - osgp-adapter-ws-db: Contains repositories for persistence.
Public Lighting - osgp-adapter-ws-publiclighting: Contains the Public Lighting web services.
Smart Metering - osgp-adapter-ws-smartmetering: Contains the Smart Metering web services.
Tariff Swithcing - osgp-adapter-ws-tariffswitching: Contains the Tariff Switching web services.
For a description of the WSDL's see the .
A description of the general package structure of a web service component.
config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.
-- ApplicationContext
-- AdapterInitializer
-- MessagingConfig
-- PersistenceConfig
-- WebServiceConfig
EndPoints for the web services: contain a reference to a service that proceeds with handling the request.
jms: contains the JMS classes such as:
-- MessageSender(s)
-- MessageType
-- ResponseMessageFinder
The WSDL and schema definitions can be found under main/webapp/WEB_INF/wsdl.
This chapter gives a more technical overview. It describes each layer of the platform by giving an overview of its packages and the code it contains. Furthermore it describes how a message proceeds through the platform. If you are planning on adding your own Domain Adapter or Protocol Adapter, it will be useful to read this chapter to get a feeling of how the Platform has been built.
The picture below depicts an example of a request (OSLP SetLight request) proceeding through the platform to a device.
A web request enters the platform at its EndPoint, which in turns calls the RequestService. The RequestService checks if the organisation in the request is authorized, creates the request message and sends it to the MessageSender which in turn puts it on the queue of the Domain Adapter.
In the Domain Adapter, the incoming message is processed in the MessageProcessor, which in turn calls the Request Service. Here the message is converted to a DTO object. The CoreRequestMessageSender puts the message on the Core Queue.
The MessageListener in Core receives the message. The DeviceRequestMessageService contains generic functionality such as Authorization, Validation, etc. Once these procedures are completed, the message is routed to the appropriate protocol adapter.
In the Protocol Adapter the message is received by the MessageListener. It is processed through the MessageProcessor and OslpDeviceService. The request eventually ends up in the OslpChannelHandler, where the actual Protocol Request to the device is made.
For a detailed description of each layer, please take a look at a more detailed description of each layer in this chapter.
The Platform uses property files for certain settings (such as JMS settings, Persistence settings, etc.). These files are stored in property files which can be found in the Config repository on Github. These files are sym linked to /etc/osp/, where the Platform (through reference in context.xml) looks for the property files.
Apache ActiveMQ: Open source messaging server, used to relay messages between components of the open smart grid platform. ActiveMQ is an open source message broker written in Java and a full Java Message Service (JMS) client. It provides "Enterprise Features" which in this case means fostering the communication from more than one client or server.
: Web server, used as front for Apache Tomcat.
: Provides a "pure Java" servlet container for Java code to run in.
: PostgreSQL administration and management tools.
: A language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.
: Agile database migration framework for Java
: JDBC connection pool
: Object/Relational mapping
: Network application framework for protocol servers & clients
: Library implementing the IEC61850 and DLMS/COSEM communication standard
: Java bean mapping
: Application development framework. Several Spring libraries are used, including Spring Data, Spring Security and Spring WS.
: Application for Automatically delivering, operating and securing your infrastructure
: Package manager for Javascript packages. Web applications consist of various components; frameworks, libraries, assets, utilities, and rainbows. Bower manages all these things for you.
: IDE for developing software.
: FTP application.
: Application designed to load test functional behaviour and measure performance.
: automated acceptance testing framework.
: DSL for acceptence testing framework.
The following table presents an overview of the components and the most important technical choices per component.
The Protocol Adapters are responsible for the actual communication to and from a device. They usually operate in a certain domain, and might use common/ generic functions from the platform.
The Open Smart Grid Platform currently has the following protocol adapters:
Protocol-Adapter-DLMS: Smart Metering
Protocol-Adapter-IEC61850: Public Lighting, Micro Grids and Distribution Automation
Protocol-Adapter-OSLP: Public Lighting and Micro Grids
config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.
-- ApplicationContext
-- MessagingConfig
-- OsgpProtocolAdapterOslpInitializer
-- OslpConfig
-- OslpPersistenceConfig
Contains the Protocol Adapter Objects used for the Protocol Adapter.
requests: Objects representing the requests that are supported by this adapter.
responses: Objects representing the responses that are supported by this adapter.
entities: Entities used for persistence.
repositories: Repositories used for persistence.
Contains the exceptions that might be thrown while communication with a device, e.g. ValidationException, MessageRejectedException, etc.
This package contains all the code for communication through JMS (Platform) and Networking (Device).
messaging: Contains the JMS Messages, MessageListeners, MessageSenders and MessageProcessors.
networking: Contains the classes that are responsible for connecting to a device using a certain protocol.
Services used to check the request status.
Throttling is used to limit the flow into communication channels that need it. The GXF platform has throttling support for CDMA and SMS/texting.
The CDMA network uses base transceiver stations (BTS) with cells for each BTS.
Throttling occurrs on the number of concurrent connections per BTS/cell. GXF offers a throttling service that hands out permits per BTS/cell while maintaining the configurable limit.
An application will typically use this service like this:
SMS can be used to wake up certain types of devices. Throttling in this context means limiting the number of messages per second to a configurable rate.
mapping: Custom Orika converters.
services: Contains the (functional) services that control communication from (and to) the device. Also persists requests and responses, and deals with security. Actual communication is done through the classes in the infra package.
Microgrids - osgp-adapter-ws-microgrids: Contains the Micro Grids web services.
Distribution Automation - osgp-adapter-ws-distributionautomation: Contains the Distribution Automation web services.
exceptionhandling: Exceptions are defined here.
mapping: Custom Orika converters.
services: Contains services used by the domain, such as AdHocManagement. These are called by the end points and convert the request to a Domain Object and put the request on the domain message queue using the JMS classes.
Git: Version control system.
NodeJS: Tooling suite with various Javascript tools.
NPM: Package manager for the NodeJS Javascript applications.
Putty: A free and open-source terminal emulator, serial console and network file transfer application.
Vim: Source code editor.
Apache Maven: Software project management tool.
GIT & GitHub: Source code management.
Sonarqube: Quality management platform.
SoapUI: Functional testing tool for testing web services.
JUnit: Unit testing.
Mockito: A Mock framework for Unit testing.
Component
Technology
Open Smart Grid Platform
Java, Spring Framework, Hibernate, Netty
Demo application
Java, Spring Framework, Spring MVC
Web services
SOAP, WSDL
OSLP Protocol
Google Protocol Buffers
Component (not open source)
Technology
OSGP Management application
Java, Spring Framework, Spring MVC
Net-Management application
Java, Spring Framework, JAX-RS, AngularJS
Liander Installatie application
Java, Spring Framework, JAX-RS, AngularJS
This chapter contains the general architecture and properties of the Open Smart Grid Platform. Domain and protocol specific information can be found in the domain and protocol chapters. This chapter is written for (potential) users, architects and developers.
The non-functional view is an overview of the most significant non-functional demands.
The identified non-functional demands are:
Time Behavior
Extensibility
The Open Smart Grid Platform is designed for message based communication.
Acts as a connecting link between (web)applications and smart devices
The open Source approach prevents vendor lock-in
Internationalisation and localisation
Security
Scalability
Fully scalable, dynamically scaling up and down as more devices and applications are added.
Freedom of choice in the desired IP communication infrastructure , e.g. CDMA and GPRS.
Stimulates open innovation by using open standards and open source technology
Multiple devices and communication protocols are supported
Independent of (cloud) hosting infrastructure
By de-linking the chain and the use of open standards and the open source license, anyone can build his or her applications on top of the open smart grid platform.
The open smart grid platform is optimized to provide reliable and efficient delivery of command and control information for e.g. smart meters, direct load control modules, solar panels, gateways and other applications.
The open smart grid platform simplifies the implementation of smart devices resulting in a shorter time-to-market by having built-in device management features
The platform supports various IP data communication infrastructures to communicate with the devices (internet, lan, GPRS, CDMA, UMTS, etc.).
The open smart grid platform also supports authentication and encryption for all data exchanges to protect the integrity and privacy of data as required in e.g. the smart grid.
The open smart grid platform supports multiple protocols
Easy application integration
Supports active-active setup over multiple data centers
Adding servers can be done in runtime
Please note: the Open Smart Grid Platform is not built for streaming data such as video, audio or a stream of high frequency measurement data.
The Open Smart Grid Platform is unique due to its multi-dimensional, generic and open design. Because of a true separation of layers and the use of open standards, other suppliers and/or third parties are able to develop and market innovative solutions.
The platform is multi-dimensional. This means that several customer use cases (with separate business models) are able to use the various device functions. One single application could use the same function of different devices.
The generic design ensures that the platform can be used in a flexible way for several functions and applications (e.g. public lighting services and smart meter services).
The platform is aimed at the 'common parts' of the technology chain; suppliers or vendors (of both applications and devices) have no competitive advantage in delivering these kind of services.
The platform layers are truly separated by open standards and the platform is made available as open source software.
The platform does not store any application data (the platform is thus stateless ). No messages/commands will ever get lost.
This enables third party vendors and developers to deliver innovative applications which are competitive in both rich functionalities and the generated data.
The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.
This chapter describes the possibilities of a redundant set up of the Open Smart Grid Platform. The Platform is designed to run in a High Available (or HA) environment, and to prevent data loss due to unexpected failures. Each component of the Platform is designed to be stateless. Components communicate with each other using message queues, which are processed in an asynchronous way.
In an active-active setup, multiple instances of each component (eg. Web Services, Core, Protocol Adapter) process the data at the same time. Traffic is equally distributed across the instances. In case of a defect in one instance the traffic is automatically processed by the remaining instance(s). The Open Smart Grid Platform is designed to run in such a set up, and thus preventing down time in case of a failing server. Each component of the Platform can run in an independent, redundant and scalable way.
The Open Smart Grid Platform environment consists of five layers:
Web services layer
Domain logic layer
Open Smart Grid Platform Core layer
Protocol layer
Device layer
In this layer the web services are exposed to the outside world. Applications can connect to the web services to implement the required functionality of the open smart grid platform. The web services are divided into functional domains, i.e. Public Lighting, Smart Metering, Power Quality, etc. Additional functional domains can be created.
Every functional domain has a separate set of web services and a corresponding domain logic block. In the domain logic block the business logic of that functional domain can be found. This is where a functional command will be translated into a generic intermediate format. For example, in the case of public lighting the command "Turn light on" will be translated into a command like "set switch(1) in closed position". In this layer it could also be decided that one functional command results in multiple commands to a device. The domain logic is closely related to the web services layers and can be added as well.
In the core of the Open Smart Grid Platform the following generic functions are found:
Device management
Time synchronization
Firmware management
Workflow engine
Device installation services
Scheduler
Device status monitoring
Routing of device commands to appropriate device protocol
The different protocol adapters are found in this layer. Here the generic intermediate format of a command for a specific device will be translated into the protocol message the device understands. This message will be sent to the device. A retry mechanism has been implemented to prevent communication failure in the case that the receiving end is temporarily unavailable. The listeners for messages initiated by a device are implemented here. Examples are the DLMS/COSEM protocol adapter for smart meters.
Any device in the public space with an Internet connection may be connected to the platform. The platform is independent of the device used, therefore this part of the set-up is not part of the platform.
Application layer: responsible for executing system tasks including authorization control
Domain layer: responsible for the representation of the problem domain.
Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc
Layers:
The web server is configured with a SSL certificate to encrypt the incoming and outgoing communication. The SOAP Web service (Spring Framework web service) uses a Java Keystore and a certificate for each organization. Only organizations that are known within the platform are authorized to use the web service.
For the several functional domains separate SOAP Web services are offered. This separation offers authorization per functional domain. Each of the web service components send a queue message to the corresponding domain component.
WSDL A separate WSDL is implemented for each functional cluster. All SOAP operations have a request object parameter and return a response object. For Synchronized Web Services the result is immediately included in the response. For asynchronous web services the response contains a correlation ID. This Correlation ID is to be used by the requester to receive the actual result from the platform. The following diagram is an example of such an asynchronous request.
Furthermore each SOAP message has a header which contains the user's organisation ID. This table displays an overview of the WSDL's including operations and fields in the request and response objects.
SOAP vs. REST
SOAP is chosen in the open smart grid platform web services over REST for the following reasons:
REST is resources/data oriented (put, get, delete) while the open smart grid platform is function/method oriented
SOAP has the advantage of having a contract (WSDL)
SOAP has extensive security features that are being used in the open smart grid platform to meet the high security demands/requirements requested by e.g. the energy utilities
Energy companies are generally not progressive in terms of technology. SOAP is acceptable for energy companies and REST is sometimes seen as new and insecure.
The benefits of REST (e.g. speed / less overhead) does not outweigh the benefits of SOAP. More general information on this topic can be found online.
For each functional domain business logic is implemented using a separate domain component. Common functionality like authorization should be abstracted to a shared component. Domain components receive queue messages from web service components and send queue messages to the open smart grid platform core component.
More information on the specific domains can be found in the domain chapter
The open smart grid platform core component receives queue messages from domain components. These messages from domain components are forwarded to a protocol adapter project. The open smart grid platform core component also offers logic for a protocol adapter project to send the response of a smart device back to a domain project. The Core component routes messages from domain adapter components to protocol adapter components and vice versa. The core layer also contains a workflow engine.
The internal database model in the core layer:
ERD's made with Valetina Studio
Overview of platform data model:
Data model explanation:
Table
Description
devices
Devices table
device_authorisation
Authorisation table, function group column concerns the device functions (AD_HOC, INSTALLATION, etc)
organization
Organization table, function group column concerns the platform functions (ADMIN of USER)
event
Events table
oslp_log_item
Table for logging of OSLP messages.
The platform will store as little data as possible. Generic (and domain specific) devices attributes are stored in core DB.
The open smart grid platform supports multiple protocols.
OSLP (Open Street Light Protocol)
DLMS/COSEM
IEC61850
The protocols can use one of the security layers:
TLS (Transport Layer Security encryption)
SSL (Secure Sockets Layer encryption)
Other protocols can be easily added to the platform. If possible, we prefer protocols based on open standards. A comprehensive list of protocols that are currently supported can be found in the protocols chapter.
Protocol specific device attributes are stored in the protocol adapter DB
Open smart grid platform components connect to each other through message queues.
Transactions on messages to and from the queues
Messages are persisted on the queues
Queues are clustered for reliability and speed
By using queues, the open smart grid platform can be stateless
The open smart grid platform can connect to any device that supports one of the supported protocols. Smart devices can receive messages from or send messages to protocol adapter components. In case of SSLD's this is done using TCP/IP over mobile internet connections (e.g. GPRS, CDMA, etc.). The communication is encrypted using public key cryptography.
The Open Smart Grid Platform uses a PostgreSQL database. PostgreSQL supports multiple database servers. For example, a slave and master node, where the slave node continuously replicates the master node. In case the master node fails, the slave mode is triggered and will stop replicating from the master node, execute a recovery and will become the master node.
The components of the Platform communicate with each other through a Message Queue. The Open Smart Grid Platform uses Apache Active Message Queue, which makes asynchronous communication possible between components. The components can register to the queues as consumers. In case a consumer (e.g. a server running a component of the platform) is down, the message will still be consumed by the remaining consumer(s). The Message Brokers can be used as a MasterSlave. In case the Master message broker is down, you get immediate fail-over to the slave without loss of messages.
webservice monitor log item
Audit record for tracking webservice activity.
The open smart grid platform use-cases are strongly related to the open smart grid platform domains. Up-to-date information on use-cases can be found on the Grid eXchange Fabric website.
