Describes the actions as defined in SmartMeteringConfiguration.wsdl

Description

GetGetFirmwareVersionResponse returns the device firmware versions requested with the GetFirmwareVersion method. The response contains the DeviceIdentification and CorrelationUid which is received from the GetFirmwareVersion request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetAdministrativeStatus is a request to retrieve the current AdministrativeStatus setting from a device. The request needs the DeviceIdentification.

All requests have similar response behaviour which is described in ResponseMessages.

GetGetAdministrativeStatusResponse returns if the setting GetAdministrativeStatus is enabled. The response contains the DeviceIdentification and CorrelationUid which is received from the GetAdministrativeStatus request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Introduction to Grid eXchange Fabric (GXF)

Grid eXchange Fabric was formerly known as the Open Smart Grid Platform (OSGP). GXF is an open, generic, scalable and independent 'Internet of Things' platform, which enables various connected smart objects in the public space to be easily controlled and monitored. Our platform allows the use of any (web)application and with any IP communication infrastructure.

Our goal is to stimulate the development of smart and sustainable solutions. Smart devices and smart apps play a central role in the development of smart grids and smart societies. The open smart grid platform software enables you to connect to thousands of devices, control them, and monitor their performance. This is done in an open and secure way, so you can use it for your own applications and devices, thereby reducing the time to market and decreasing development costs.

The names Open Smart Grid Platform and Grid eXchange Fabric will both be found in this document and in the code. In the future we will replace all references to the Open Smart Grid Platform and OSGP by Grid eXchange Fabric and GXF. During this transition period please read Grid eXchange Fabric if you see Open Smart Grid Platform.

Using the platform

The Grid eXchange Fabric is used in the following way:

• A user or operator uses one or more (web) applications to monitor and/or control devices

• The applications connect to the open smart grid platform via several web services which are divided into functional domains, i.e. Public Lighting, Smart Metering, Power Quality, etc. Third party developers can use the web services for the development or integration of new applications

• The platform handles all these application requests in a secure way and uses various functions and services to do so (e.g. authorization, authentication, device management, logging)

• For the 'translation' and communication of user/operator commands to the various smart devices, the platform uses multiple (open) protocols

• The platform supports various IP based data telecommunication technologies and protocols to communicate with the devices

Example use case for Grid eXchange Fabric

Use cases of Grid eXchange Fabric are only limited by your imagination. Here are some examples:

• Ad-hoc and scheduled Switching of Public Lighting

• Electrical Transportation

• Smart Metering

• Traffic Control (LED/matrix signs, traffic lights)

• Flexible load management ( solar energy and wind energy)

• Power Quality monitoring

Getting started

Description

GetFirmwareVersion is a request to retrieve the firmware version(s) of a device. The request needs the DeviceIdentification.

All requests have similar response behaviour which is described in ResponseMessages.

GetGetFirmwareVersionResponse returns the version(s). The response contains the DeviceIdentification and CorrelationUid which is received from the GetFirmwareVersion request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetGetAdministrativeStatusResponse returns if the setting GetAdministrativeStatus is enabled. The response contains the DeviceIdentification and CorrelationUid which is received from the GetAdministrativeStatus request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

ReplaceKeys is a request to change the keys on an E-meter. The request needs the DeviceIdentification, an AuthenticationKey and an EncryptionKey.

All requests have similar response behaviour which is described in ResponseMessages.

GetReplaceKeysResponse returns if the result is successful from the ReplaceKeys request. The response contains the DeviceIdentification and CorrelationUid which is received from the ReplaceKeys request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

UpdateFirmware is a request to install another firmware version(s) on a device. The request needs the DeviceIdentification and the firmware versions, that together with the device model (as stored with the identified device) uniquely determine the firmware file to be used.

All requests have similar response behaviour which is described in ResponseMessages.

GetUpdateFirmwareResponse returns the version(s). The response contains the DeviceIdentification and CorrelationUid which is received from the UpdateFirmware request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetUpdateFirmwareResponse returns the device firmware versions that are on the device after calling the UpdateFirmware method. The response contains the DeviceIdentification and CorrelationUid which is received from the UpdateFirmware request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

SetActivityCalendar is a request to set tariffs on an E-meter according a SeasonProfile and WeekProfile. In a WeekProfile, seven dayprofiles can be filled in with a start time and dayId which contains the tariff.

The request needs the DeviceIdentification, CalendarName, ActivatePassiveCalendarTime, SeasonProfileName, SeasonStart, WeekProfileName, DayId and StartTime.

All requests have similar response behaviour which is described in .

returns the result from setting a SetActivityCalendar. The response contains the DeviceIdentification and CorrelationUid which is received from the SetActivityCalendar request.

References

XSD:

WSDL:

Description

SetAdministrativeStatusis a request to set the AdministrativeStatus on a device. The request needs the DeviceIdentification and Enabled parameter.

All requests have similar response behaviour which is described in .

returns if the setting SetAdministrativeStatus is enabled. The response contains the DeviceIdentification and CorrelationUid which is received from the SetAdministrativeStatus request.

References

XSD:

WSDL:

Description

SetAlarmNotifications is a request to set the types of alarm notifications that must be notified from the device when they occur. The following notifications can be enabled or disabled:

CLOCK_INVALID, REPLACE_BATTERY, POWER_UP, PROGRAM_MEMORY_ERROR, RAM_ERROR, NV_MEMORY_ERROR, MEASUREMENT_SYSTEM_ERROR, WATCHDOG_ERROR, FRAUD_ATTEMPT, COMMUNICATION_ERROR_M_BUS_CHANNEL_1, COMMUNICATION_ERROR_M_BUS_CHANNEL_2, COMMUNICATION_ERROR_M_BUS_CHANNEL_3, COMMUNICATION_ERROR_M_BUS_CHANNEL_4, FRAUD_ATTEMPT_M_BUS_CHANNEL_1, FRAUD_ATTEMPT_M_BUS_CHANNEL_2, FRAUD_ATTEMPT_M_BUS_CHANNEL_3, FRAUD_ATTEMPT_M_BUS_CHANNEL_4, NEW_M_BUS_DEVICE_DISCOVERED_CHANNEL_1, NEW_M_BUS_DEVICE_DISCOVERED_CHANNEL_2, NEW_M_BUS_DEVICE_DISCOVERED_CHANNEL_3, NEW_M_BUS_DEVICE_DISCOVERED_CHANNEL_4

The request needs the DeviceIdentification, AlarmType and Enabled parameters.

All requests have similar response behaviour which is described in .

returns the result from setting a SetAlarmNotifications. The response contains the DeviceIdentification and CorrelationUid which is received from the SetAlarmNotifications request.

References

XSD:

WSDL:

The Open Smart Grid Platform is designed for message based communication.

• Acts as a connecting link between (web)applications and smart devices

• The open Source approach prevents vendor lock-in

• State of the art security

• Fully scalable, dynamically scaling up and down as more devices and applications are added.

• Freedom of choice in the desired IP communication infrastructure , e.g. CDMA and GPRS.

• Stimulates open innovation by using open standards and open source technology

• Multiple devices and communication protocols are supported

• Independent of (cloud) hosting infrastructure

• By de-linking the chain and the use of open standards and the open source license, anyone can build his or her applications on top of the open smart grid platform.

• The open smart grid platform is optimized to provide reliable and efficient delivery of command and control information for e.g. , direct modules, , gateways and other applications.

• The open smart grid platform simplifies the implementation of smart devices resulting in a shorter time-to-market by having built-in device management features

• The platform supports various IP data communication infrastructures to communicate with the devices (internet, lan, GPRS, CDMA, UMTS, etc.).

• The open smart grid platform also supports authentication and encryption for all data exchanges to protect the integrity and privacy of data as required in e.g. the smart grid.

• The open smart grid platform supports multiple protocols

• Easy application integration

• Supports active-active setup over multiple data centers

• Adding servers can be done in runtime

Please note: the Open Smart Grid Platform is not built for streaming data such as video, audio or a stream of high frequency measurement data.

Unique features of the Open Smart Grid Platform

The Open Smart Grid Platform is unique due to its multi-dimensional, generic and open design. Because of a true separation of layers and the use of open standards, other suppliers and/or third parties are able to develop and market innovative solutions.

1. The platform is multi-dimensional. This means that several customer use cases (with separate business models) are able to use the various device functions. One single application could use the same function of different devices.

2. The generic design ensures that the platform can be used in a flexible way for several functions and applications (e.g. public lighting services and smart meter services).

3. The platform is aimed at the 'common parts' of the technology chain; suppliers or vendors (of both applications and devices) have no competitive advantage in delivering these kind of services.

4. The platform layers are truly separated by open standards and the platform is made available as open source software.

5. The platform does not store any application data (the platform is thus stateless ). No messages/commands will ever get lost.

   This enables third party vendors and developers to deliver innovative applications which are competitive in both rich functionalities and the generated data.

Functional view

Image, functional layers overview

Starting architecture

The Functional view shows an overview of the most important functions of the system. The two images below show the starting architecture and functional reference architecture respectively.

Image, functional starting architecture

1. Web applications

2. Open Smart Grid Platform

3. Web services

4. Basic functions

5. Database

6. Communication infrastructure (CDMA/GPRS/Ethernet)

7. IP infrastructure

8. Open Street Light Protocol (OSLP)

9. Public Street Lighting Device (PSLD) or Sub Station Lighting Device (SSLD)

Functional Reference

This model partitions the system in seven functional clusters (vertically) which are shown on the system layers (horizontally). The circled numbers refer to image 1.

Vertical clusters:

• Device installation

• Device management

• Firmware management

• Configuration management

• Schedule management

• Ad-hoc control and status

• Monitoring

Horizontal System layers:

• Web applications

• HTTPS/SOAP communication

• Platform

• Open protocols

• Smart devices

Description

GetSetAdministrativeStatusResponse returns if the setting SetAdministrativeStatus is enabled. The response contains the DeviceIdentification and CorrelationUid which is received from the SetAdministrativeStatus request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

This chapter contains the general architecture and properties of the Open Smart Grid Platform. Domain and protocol specific information can be found in the domain and protocol chapters. This chapter is written for (potential) users, architects and developers.

Description

GetSetActivityCalendarResponse returns the result from setting a SetActivityCalendar. The response contains the DeviceIdentification and CorrelationUid which is received from the SetActivityCalendar request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetSetConfigurationObjectResponse returns the result from setting a ConfigurationObject. The response contains the DeviceIdentification and CorrelationUid which is received from the SetConfigurationObject request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

SetConfigurationObject is a request to set ConfigurationObject settings on a device. The attributes with OBIS code 0-1:94.31.3.255 give access to set GPRS_operation_mode setting and following flags:

• discover_on_open_cover

• discover_on_power_on

• dynamic_mbus_address

• P0_enable

• HLS_3_on_P3_enable

• HLS_4_on_P3_enable

• HLS_5_on_P3_enable

• HLS_3_on_P0_enable

• HLS_4_on_P0_enable

• HLS_5_on_P0_enable

See DSMR document chapter 8.3 for detailed description. The request needs the DeviceIdentification, GprsOperationMode, ConfigurationFlagType and Enabled parameters.

All requests have similar response behaviour which is described in ResponseMessages.

GetSetConfigurationObjectResponse returns the result from setting a SetConfigurationObject. The response contains the DeviceIdentification and CorrelationUid which is received from the SetConfigurationObject request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetSetAlarmNotificationsResponse returns the result from setting a SetAlarmNotifications. The response contains the DeviceIdentification and CorrelationUid which is received from the SetAlarmNotifications request.

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

GetKeys is a request to retrieve keys of a device. Multiple keys can be requested in one request. The keys in the response will be encrypted with the configured public key of the calling application.

The following key types are allowed:

• E_METER_MASTER_KEY,

• E_METER_AUTHENTICATION_KEY,

• E_METER_ENCRYPTION_KEY_UNICAST,

• E_METER_ENCRYPTION_KEY_BROADCAST,

• G_METER_MASTER_KEY,

• G_METER_ENCRYPTION_KEY,

• G_METER_FIRMWARE_UPDATE_AUTHENTICATION_KEY,

• G_METER_OPTICAL_PORT_KEY

All requests have similar response behaviour which is described in ResponseMessages.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Description

SetKeyOnGMeter is a request to transfer and set an encryption key, firmware update authentication key or an optical port key on a G-meter via the E-meter. The request needs the DeviceIdentification from the G-meter and the key type (secrettype). If the device identification of the G-meter is not known, but the gateway device identification and M-Bus channel are known, use the SetMbusUserKeyByChannel request instead. If the key type is the optical port key, the parameter to close the optical port can be set as well.

All requests have similar response behaviour which is described in ResponseMessages.

GetSetKeyOnGMeterResponse returns the result from setting a SetKeyOnGMeter. The response contains the DeviceIdentification and CorrelationUid which is received from the SetKeyOnGMeter request.

References

XSD: sm-configuration.xsd

WSDL: SmartMeteringConfiguration.wsdl

Basic Architecture

The basic architecture

Basic Overview

Layered architecture

The Open Smart Grid Platform environment consists of five layers:

1. Web services layer

2. Domain logic layer

3. Open Smart Grid Platform Core layer

4. Protocol layer

5. Device layer

Web services layer

In this layer the web services are exposed to the outside world. Applications can connect to the web services to implement the required functionality of the open smart grid platform. The web services are divided into functional domains, i.e. Public Lighting, Smart Metering, Power Quality, etc. Additional functional domains can be created.

Domain logic layer

Every functional domain has a separate set of web services and a corresponding domain logic block. In the domain logic block the business logic of that functional domain can be found. This is where a functional command will be translated into a generic intermediate format. For example, in the case of public lighting the command "Turn light on" will be translated into a command like "set switch(1) in closed position". In this layer it could also be decided that one functional command results in multiple commands to a device. The domain logic is closely related to the web services layers and can be added as well.

Open Smart Grid Platform core layer

In the core of the Open Smart Grid Platform the following generic functions are found:

• Device management

• Time synchronization

• Firmware management

• Workflow engine

• Device installation services

• Scheduler

• Device status monitoring

• Routing of device commands to appropriate device protocol

Protocol layer

The different protocol adapters are found in this layer. Here the generic intermediate format of a command for a specific device will be translated into the protocol message the device understands. This message will be sent to the device. A retry mechanism has been implemented to prevent communication failure in the case that the receiving end is temporarily unavailable. The listeners for messages initiated by a device are implemented here. Examples are the DLMS/COSEM protocol adapter for smart meters.

Devices

Any device in the public space with an Internet connection may be connected to the platform. The platform is independent of the device used, therefore this part of the set-up is not part of the platform.

Description

GetSetKeyOnGMeterResponse returns the result from setting a SetKeyOnGMeter. The response contains the DeviceIdentification and CorrelationUid which is received from the request.

All requests have similar response behaviour which is described in .

References

XSD:

WSDL:

Description

GetSetPushSetupSmsResponse returns the result from setting a SetPushSetupSms. The response contains the DeviceIdentification and CorrelationUid which is received from the request.

All requests have similar response behaviour which is described in .

References

XSD:

WSDL:

Description

SetPushSetupAlarm is a request to define the TCP message that is optionally sent by the device. The request consists of the DeviceIdentification and at least one of the following optional items:

• The destination: Host URL and port.

• The push object list, defining which information should be sent in the alarm.

If an item is not included in the message, the value in the meter will remain unchanged.

All requests have similar response behaviour which is described in .

returns the result from setting a SetPushSetupAlarm. The response contains the DeviceIdentification and CorrelationUid which is received from the SetPushSetupAlarm request.

References

XSD:

WSDL:

Description

SetSpecialDays is a request to set a dayId profile for a specific date on a device, other than the standard applicable dayId's. This can be useful to change tariffs and tariff scheduling for specific days such as public holidays. The request is send with the DeviceIdentification number from the desired device, date and dayId.

All requests have similar response behaviour which is described in .

returns the result from setting a Special Day. The response contains the DeviceIdentification and CorrelationUid which is received from the SetSpecialDays request.

References

XSD:

WSDL:

Description

SetPushSetupSms is a request to set an endpoint in a device which tells the device where to connect to when it is woken. The request needs the DeviceIdentification, host URL and port.

All requests have similar response behaviour which is described in .

returns the result from setting a SetPushSetupSms. The response contains the DeviceIdentification and CorrelationUid which is received from the SetPushSetupSms request.

References

XSD:

WSDL:

Description

GetSetPushSetupAlarmResponse returns the result from setting a SetPushSetupAlarm. The response contains the DeviceIdentification and CorrelationUid which is received from the request.

All requests have similar response behaviour which is described in .

References

XSD:

WSDL:

This chapter gives an overview of the principles used defining and implementing the architecture. The following principles were applied:

• Layering

• Domain driven design

• Dependency inversion principle

• Behavior driven development

Layering

The use of layers improves the separation of responsibilities. Each application contains the following layers:

• Presentation layer: responsible for providing information to users (persons and/or systems) and the handling of user requests

• Application layer: responsible for executing system tasks including authorisation control

• Domain layer: responsible for the representation of the problem domain.

• Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc

Image, Layers:

1. Audit logger

2. Web Services

3. Functions

4. Queue

5. Workflow engine

6. Protocol framework

7. Protocol implementations

8. Workflow engine

9. Queue

10. Communication

Domain driven design (DDD)

Domain-driven design focuses on the problem domain. DDD's starting point is creating an optimal model for a specific problem domain by having a common language and constructive collaboration between technical and domain experts.

DDD uses the following building blocks:

• Entity: An object not identified by its attributes but by its own identity.

• Value Object: an object with attributes but has no own identity.

• A collection of objects surrounding a specific root entity (or aggregate root). To ensure consistency objects in the aggregate can only be addressed through the aggregate root.

• Service: Contains instructions not related to a specific object.

• Repository: Serves as a collection for fetching and saving objects. Creates an abstraction for actual persistent implementations.

• Factory: Contains methods to create domain objects.

Dependency inversion principle

The dependency inversion principle promotes an independent connection by inverting dependency relations. This ensures that the domain model can be very 'clean' without knowledge of the underlying infrastructure (POJO classes). The Spring framework is used to implement the Dependency Inversion principle.

Behavior driven development (BDD)

Behavior driven development is a way of programming that first describes behavior in user stories and then implements this in code. The user stories contain scenarios with acceptation criteria that can be automated. This creates a complete test suite for the whole system.

For the application of BDD the following frameworks are used:

• Cucumber and Gherkin, automated acceptance testing, based on scenarios from stories.

Time behavior is mainly important in the Flexovl application when a lot of devices have to be addressed in a short period of time over a wireless network. Both latency and limited bandwidth have to be taken into consideration while demanding the coordinated on and off switching of the lighting, since we want to avoid the Christmas tree effect.

• Time synchronization: devices periodically register with the platform and receive a time.

• Protocol: because of the limited bandwidth an efficient protocol "protobuf" was selected.

Points of interest:

• Light metering messages

• When the SSLD's are disabled the PSLDs cannot be addressed

Because of these points of interest we use message queueing combined with a retry mechanism of delayed delivery.

The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.

The non-functional view is an overview of the most significant non-functional demands.

The identified non-functional demands are:

• Time Behavior

• Extensibility

• Internationalisation and localisation

• Security

• Scalability

This are some examples how a message flows in the Open Smart Grid Platform.

Message Flow: Request/Acknowledge/Poll

Message Flow: Request/Acknowledge/Notify

*In case the response is not timely retrieved by the client app, OSGP will resend the notification with correlation id to the client app. The amount of retries is configurable.

Application Layering

The use of layers improves the separation of responsibilities. Each application contains the following layers:

• Presentation layer: responsible for providing information to users (persons and/or systems) and the handling of user requests

• Application layer: responsible for executing system tasks including authorization control

• Domain layer: responsible for the representation of the problem domain.

• Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc

Layers:

Authentication and authorization

The web server is configured with a SSL certificate to encrypt the incoming and outgoing communication. The SOAP Web service (Spring Framework web service) uses a Java Keystore and a certificate for each organization. Only organizations that are known within the platform are authorized to use the web service.

Application integration layer

For the several functional domains separate SOAP Web services are offered. This separation offers authorization per functional domain. Each of the web service components send a queue message to the corresponding domain component.

WSDL A separate WSDL is implemented for each functional cluster. All SOAP operations have a request object parameter and return a response object. For Synchronized Web Services the result is immediately included in the response. For asynchronous web services the response contains a correlation ID. This Correlation ID is to be used by the requester to receive the actual result from the platform. The following diagram is an example of such an asynchronous request.

Furthermore each SOAP message has a header which contains the user's organisation ID. This table displays an overview of the WSDL's including operations and fields in the request and response objects.

SOAP vs. REST

SOAP is chosen in the open smart grid platform web services over REST for the following reasons:

• REST is resources/data oriented (put, get, delete) while the open smart grid platform is function/method oriented

• SOAP has the advantage of having a contract (WSDL)

• SOAP has extensive security features that are being used in the open smart grid platform to meet the high security demands/requirements requested by e.g. the energy utilities

• Energy companies are generally not progressive in terms of technology. SOAP is acceptable for energy companies and REST is sometimes seen as new and insecure.

The benefits of REST (e.g. speed / less overhead) does not outweigh the benefits of SOAP. More general information on this topic can be found online.

Domain logic layer

For each functional domain business logic is implemented using a separate domain component. Common functionality like authorization should be abstracted to a shared component. Domain components receive queue messages from web service components and send queue messages to the open smart grid platform core component.

More information on the specific domains can be found in the domain chapter

Open Smart Grid Platform Core Services

The open smart grid platform core component receives queue messages from domain components. These messages from domain components are forwarded to a protocol adapter project. The open smart grid platform core component also offers logic for a protocol adapter project to send the response of a smart device back to a domain project. The Core component routes messages from domain adapter components to protocol adapter components and vice versa. The core layer also contains a workflow engine.

ERD's made with Valetina Studio

Overview of platform data model:

Data model explanation:

The platform will store as little data as possible. Generic (and domain specific) devices attributes are stored in core DB.

Protocol Layer

The open smart grid platform supports multiple protocols.

• OSLP (Open Street Light Protocol)

• DLMS/COSEM

• IEC61850

The protocols can use one of the security layers:

• TLS (Transport Layer Security encryption)

• SSL (Secure Sockets Layer encryption)

Other protocols can be easily added to the platform. If possible, we prefer protocols based on open standards. A comprehensive list of protocols that are currently supported can be found in the protocols chapter.

Protocol specific device attributes are stored in the protocol adapter DB

Queues

Open smart grid platform components connect to each other through message queues.

• Transactions on messages to and from the queues

• Messages are persisted on the queues

• Queues are clustered for reliability and speed

• By using queues, the open smart grid platform can be stateless

Smart devices

The open smart grid platform can connect to any device that supports one of the supported protocols. Smart devices can receive messages from or send messages to protocol adapter components. In case of SSLD's this is done using TCP/IP over mobile internet connections (e.g. GPRS, CDMA, etc.). The communication is encrypted using public key cryptography.

The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.

Security

The following security measures can be used in a hosted environment:

Cloud security

• DDOS protection

• IPSEC VPN connections

• IP whitelisting

Most cloud environments support these features.

Operating System

• Hardened operating systems (according to Center of Internet Security)

Platform security

• Communication over TLS

• Firewalls between all servers and layers

• Certificates from a recognized Certificate Authority (CA)

• Audit trail on all actions throughout the platform

• Role based authorizations on specific functions of devices

• Access control

• Unique device identification

For every major release there will be a mandated security test initiated by Alliander.

In cooperation with the European Network of Cyber Security (ENCS) state of the art security measures were implemented.

• Security per device

• Security per application

• Security Certificates per Organisation and per device

• All communication is encrypted

Security measures: 1. Firewall in defined zone 2. Operating System Hardening 3. DDOS protection 4. Replay attack prevention 5. Private encryption key per device 6. Certificates from a Certificate Authority 7. Encryption via Elliptic Curve DSA 8. IPSEC VPN for CDMA and GPRS 9. Unique device identification 10. Unique CDMA modem number 11. Role based authorizations on functions and devices

Encryption

An analysis of safety aspects has led to the decision that the safety of the whole system will be realized by proven technology based on asymmetrical coding (also known as public-key encryption).

Authentication of web applications

Two-way SSL will be used between web applications and the Open Smart Grid Platform to verify the identities for both client and server. User organisations are responsible for the administration of the identity of and access to their web applications. The web applications feature a login page. After successful login the user is linked to an organisation. Passwords will be stored with encryption. The organisation ID will be sent in each message to the Open Smart Grid Platform and will be verified by the SSL certificate.

Algorithms

Only public encryption Algorithms will be used. Due to performance limitations (of the devices) and recommendations from The European Network for Cyber Security (ENCS) Elliptic Curve DSA with 256-bit-keys was selected. This improves the security and efficiency over the 1024 bit RSA algorithm. Messages can be smaller and less processor capacity is needed. The key length of Elliptic Curve DSA is similar to the 3072 bit key length of RSA.

Note: Even though the open smart grid platform uses ECDSA to secure the OSLP, other encryptions may be used as well. The RSA Algorithm is still supported if preferred. This is a flexible configuration option.

Private APN

A private APN is used for linking to mobile data communication infrastructures.

Logging

• Every action to and from devices is logged in the audit trail

• Messages from unknown devices will be denied (and logged)

Authentication of open smart grid platform

The Open Smart Grid Platform contains an extensive authorization model, which enables a device owner to give certain rights on certain devices to other organizations. Every organization will only see devices they have rights to.

This model displays the most important entities of the open smart grid platform system and their mutual relationships.

Image of Logical Authorisation Data Model

The logic of the model above:

At the top of the image is the entity "authorisation". This represents the permissions of an organization on a certain device. In general an organisation will have a lot of permissions, at least one for each device it needs to manage.

The functions an organisation can execute on a device are determined by the function group the authorisation refers to. Function groups are collections of functions and are predefined in the software. The following function groups have been predefined.

• Owner-group (this contains all functions)

• Ad hoc-group (Functions for ad hoc switching of lighting)

• Management-group (Platform functions)

• Installation-group (Functions to install devices)

• Firmware-group (Functions for updating firmware)

• Schedule-group (Functions to create lighting schedules)

• Tariff scheme-group (Functions to create tariff groups)

• Configuration-group (Functions to configure devices)

• Monitoring-group (Functions to monitor devices)

This structure provides maximum flexibility when assigning rights to devices. Devices always belong to an Owner. An owner is an organisation, but not every organisation is an owner. A device can have more than one owner. The entity "Event", at the bottom of the image, is the execution of a function by an organisation on a device.

Details like device-type, device-status, etc. have been omitted from this model.

One security requirement is that each event must be traced back to a 'natural' person, also known as an audit trail. Although the open smart grid platform does not register individual users we can meet this requirement by registering a data-item with each event. This enables the user organisation to investigate which events belongs to which 'natural' person. This data-item can for example be an user-ID provided by the user organisation which doesn't have to be unique in the open smart grid platform.

Table describing the entities in the logical data-model

An organization can get rights to one or more function groups, and thus all functions in that function group will be available to this organization.

To ensure that devices can only receive instructions from a 'genuine' open smart grid platform it must be possible to authenticate the open smart grid platform. This is implemented through a standard technology based on asymmetric encryption (if supported by the Device). The open smart grid platform will receive a unique key to enable the devices to tell if the messages come from a 'genuine' open smart grid platform. Both OSLP and DLMS device types use this kind of encryption. To prevent replay-attacks each message will get an index number (this is standard practice as well).

Authentication of devices

To ensure that the open smart grid platform can distinguish between 'genuine' devices and 'illegal' devices, all devices are supplied with a manufacturer key. Each device has a unique key. Because of the asymmetrical encryption the platform contains the public part of each key. In this way devices can be identified by their unique key and their unique hardware ID. The device-ID will be encrypted in each message sent from the device to the platform.

All communication between the open smart grid platform and the devices will be signed with these keys to ensure (1) the source is legitimate and (2) to ensure the integrity of the message. It is not necessary to encrypt the whole message because confidentiality is not important. This results in a less computationally intensive process.

When a key is stolen (by hacking a device) this will not affect the integrity of the other devices. Each device has an unique key after all and only the hacked device has to be excluded from communication in the platform.

The security is independent from the carrier (GPRS, CDMA, Ethernet, etc.). The open smart grid platform supports symmetric and asymmetric encryption (depends on device and protocol).

For OSLP devices, the firmware will be used to distribute keys to devices. In this way we can use the existing secure firmware update mechanism for updating keys and certificates. DLMS devices use a mechanism to switch keys that is not dependent on firmware updates.

Additional security may be provided by using TLS communication.

Authorisation of organisations

Authorisation for use of the platform functionalities is handled by function groups. Function groups are defined for both platform functionality and device functionality. Each function group has one or more functions. Access to device functions can be set per device. The tables below show an overview of all function-groups and device-functions and platform-groups and platform-functions respectively.

The Open Smart Grid Platform is designed and built for scalability and reliability

• Messages will never get lost, In the worst case scenario, a message will be sent to the dead letter queue.

• Any layer of the platform can be independently scaled up- and down

• Adding servers can be done runtime

• It can run in an active-active setup over multiple servers and data centers. In our cloud hosted setup even over sets of data centers in different countries.

This chapter describes the results of a performance test, to give potential users an indication of the system requirements for the platform.

The Platform was tested with the following AWS setup:

Systems:

• Specifications Component Server: 2 CPU's, 8 GB RAM

• Specifications Database Server: 1 CPU, 2 GB RAM

Setup:

• Front-end: 2x Component Server

• Middle-end: 2x Component Server

• Back-end: 2x Component Server

• ActiveMQ Front-Middle: 1x Component Server

• ActiveMQ Middle-Back: 1x Component Server

• Databases: 1x DB Server

For the test to succeed, two requirements were to be met:

1. Switch 10.000 simulated OSLP devices under 5 minutes.

2. Switch 40.000 simulated OSLP devices under 5 minutes.

The results showed that both tests succeeded.

This chapter describes the possibilities of a redundant set up of the Open Smart Grid Platform. The Platform is designed to run in a High Available (or HA) environment, and to prevent data loss due to unexpected failures. Each component of the Platform is designed to be stateless. Components communicate with each other using message queues, which are processed in an asynchronous way.

Active-active

In an active-active setup, multiple instances of each component (eg. Web Services, Core, Protocol Adapter) process the data at the same time. Traffic is equally distributed across the instances. In case of a defect in one instance the traffic is automatically processed by the remaining instance(s). The Open Smart Grid Platform is designed to run in such a set up, and thus preventing down time in case of a failing server. Each component of the Platform can run in an independent, redundant and scalable way.

Database

The Open Smart Grid Platform uses a PostgreSQL database. PostgreSQL supports multiple database servers. For example, a slave and master node, where the slave node continuously replicates the master node. In case the master node fails, the slave mode is triggered and will stop replicating from the master node, execute a recovery and will become the master node.

ActiveMQ

The components of the Platform communicate with each other through a Message Queue. The Open Smart Grid Platform uses Apache Active Message Queue, which makes asynchronous communication possible between components. The components can register to the queues as consumers. In case a consumer (e.g. a server running a component of the platform) is down, the message will still be consumed by the remaining consumer(s). The Message Brokers can be used as a MasterSlave. In case the Master message broker is down, you get immediate fail-over to the slave without loss of messages.

The Domain Adapters are responsible for receiving requests from the Web Services layer, and delivering them to the Core layer. The Domain Layer mainly contains MessageProcessors and Services for request handling.

The Core/Admin components contains the shared functionality, while the Domain components contain additional domain specific functionality.

At the moment the Platform uses the following Domain Adapters:

Generic

• Core - osgp-adapter-domain-core: Contains Core (common) functionality; AdHocManagement, FirmwareManagement, etc.

• Admin - osgp-adapter-domain-admin: Contains Admin functionality, e.g. DeviceManagement.

Domain

• Public Lighting - osgp-adapter-domain-publiclighting: Contains functionality for the Public Lighting Domain.

• Smart Metering - osgp-adapter-domain-smartmetering: Contains functionality for the Smart Metering Domain.

• Tariff Switching - osgp-adapter-domain-tariffswitching: Contains functionality for the Tariff Switching Domain.

• Microgrids - osgp-adapter-domain-microgrids: Contains functionality for the Micro Grids domain.

• Distribution Automation - osgp-adapter-domain-distributionautomation: Contains functionality for the Distribution Automation domain.

General Package structure

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- DomainAdapterInitializer

  -- MessagingConfig

  -- PersistenceConfig

• mapping: Custom Orika converters for mapping to/from DomainObjects/DTO Objects.

• services: Contains most of the domain logic, related to the specific services of the adapter. The service classes converts DTO objects to Domain objects (or vice versa), and put the request on the Core queue through the JMS classes.

infra

• jms.core: Inbound/outbound messages from/to the Core layer.

  This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Core Queue, or receiving and processing responses from Core.

• jms.ws: Inbound/outbound messages from/to the web services layer.

  This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Web Services Queue, or receiving and processing responses from the web services layer.

The Web Services layer contains the web services that are used to communicate with the Platform. The Open Source Smart Grid Platform uses the Simple Object Access Protocol or SOAP to expose its interfaces. The Web Services Adapter receives requests and sends those to the Domain Adapter. An incoming request is converted to a Domain Object, and put on the MessageQueue of the Domain layer. The Web Services layer also has a queue for incoming responses from the Domain adapter.

Each domain of the Platform has its own web services:

Generic

• Core - osgp-adapter-ws-core: Contains the Core (common) web services.

• Admin - osgp-adapter-ws-admin: Contains the Admin web services.

• Shared - osgp-adapter-ws-shared: Contains shared endpoints, such as header authorization

• Database - osgp-adapter-ws-db: Contains repositories for persistence.

Domain

• Public Lighting - osgp-adapter-ws-publiclighting: Contains the Public Lighting web services.

• Smart Metering - osgp-adapter-ws-smartmetering: Contains the Smart Metering web services.

• Tariff Swithcing - osgp-adapter-ws-tariffswitching: Contains the Tariff Switching web services.

• Microgrids - osgp-adapter-ws-microgrids: Contains the Micro Grids web services.

• Distribution Automation - osgp-adapter-ws-distributionautomation: Contains the Distribution Automation web services.

For a description of the WSDL's see the Domain Chapter.

General Package structure

A description of the general package structure of a web service component.

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- AdapterInitializer

  -- MessagingConfig

  -- PersistenceConfig

  -- WebServiceConfig

• exceptionhandling: Exceptions are defined here.

• mapping: Custom Orika converters.

• services: Contains services used by the domain, such as AdHocManagement. These are called by the end points and convert the request to a Domain Object and put the request on the domain message queue using the JMS classes.

endpoints

• EndPoints for the web services: contain a reference to a service that proceeds with handling the request.

infra

• jms: contains the JMS classes such as:

  -- MessageSender(s)

  -- MessageType

  -- ResponseMessageFinder

webapp

The WSDL and schema definitions can be found under main/webapp/WEB_INF/wsdl.

The Core layer of the Open Source Grid Platform is responsible for Validation, Translation, Authorisation and Routing of request messages. It also contains all the Domain Objects.

The core layer consists of two components:

• osgp-domain-core: Shared Domain objects, services, repositories, etc. These classes are used through the entire platform.

• osgp-core: Logic for routing domain requests, scheduling, retrying, etc.

General Package structure: osgp-domain-core

• entities: Defines the entities used for persistence.

• exceptions: Domain specific exceptions reside here.

• repositories: Repositories that contain logic for persisting entities.

• services: Domain services that reference a repository.

• specifications: Interfaces that define specifications for Devices and Events.

• validations: Validators and constraints.

• valueobjects: Definitions of the Domain Objects.

General Package structure: osgp-core

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- OsgpCoreInitializer

  -- DomainMessagingConfig

  -- PersistenceConfig

  -- ProtocolMessagingConfig

  -- SchedulingConfig

• services: Services that process device requests/ responses. Checks for authorization, and if the request is supported by the platform, it will be routed to the appropriate protocol adapter.

• tasks: Contains task scheduler logic.

domain.model

These packages contain interfaces for the Services.

• domain: Interfaces for the Domain services.

• protocol: Interfaces for the Protocol services.

infra.jms

• domain: Contains Messages, MessageListeners and MessageProcessors for Domain related messaging.

• protocol: Contains Messages, MessageListeners and MessageProcessors for Protocol related messaging.

The Protocol Adapters are responsible for the actual communication to and from a device. They usually operate in a certain domain, and might use common/ generic functions from the platform.

The Open Smart Grid Platform currently has the following protocol adapters:

• Protocol-Adapter-DLMS: Smart Metering

• Protocol-Adapter-IEC61850: Public Lighting, Micro Grids and Distribution Automation

• Protocol-Adapter-OSLP: Public Lighting and Micro Grids

General package structure

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- MessagingConfig

  -- OsgpProtocolAdapterOslpInitializer

  -- OslpConfig

  -- OslpPersistenceConfig

• mapping: Custom Orika converters.

• services: Contains the (functional) services that control communication from (and to) the device. Also persists requests and responses, and deals with security. Actual communication is done through the classes in the infra package.

device

Contains the Protocol Adapter Objects used for the Protocol Adapter.

• requests: Objects representing the requests that are supported by this adapter.

• responses: Objects representing the responses that are supported by this adapter.

domain

• entities: Entities used for persistence.

• repositories: Repositories used for persistence.

exceptions

Contains the exceptions that might be thrown while communication with a device, e.g. ValidationException, MessageRejectedException, etc.

infra

This package contains all the code for communication through JMS (Platform) and Networking (Device).

• messaging: Contains the JMS Messages, MessageListeners, MessageSenders and MessageProcessors.

• networking: Contains the classes that are responsible for connecting to a device using a certain protocol.

services

Services used to check the request status.

Platform

• Apache ActiveMQ: Open source messaging server, used to relay messages between components of the open smart grid platform. ActiveMQ is an open source message broker written in Java and a full Java Message Service (JMS) client. It provides "Enterprise Features" which in this case means fostering the communication from more than one client or server.

• Apache HTTP server: Web server, used as front for Apache Tomcat.

• Apache Tomcat: Provides a "pure Java" servlet container for Java code to run in.

• pgAdmin-III: PostgreSQL administration and management tools.

• Protobuf (Google Protocol Buffers): A language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.

• Flyway: Agile database migration framework for Java

• HikariCP: JDBC connection pool

• Hibernate: Object/Relational mapping

• Netty: Network application framework for protocol servers & clients

• OpenMUC: Library implementing the IEC61850 and DLMS/COSEM communication standard

• Orika: Java bean mapping

• Spring: Application development framework. Several Spring libraries are used, including Spring Data, Spring Security and Spring WS.

• Puppet: Application for Automatically delivering, operating and securing your infrastructure

Development

• Bower: Package manager for Javascript packages. Web applications consist of various components; frameworks, libraries, assets, utilities, and rainbows. Bower manages all these things for you.

• Eclipse: IDE for developing software.

• FileZilla: FTP application.

• Git: Version control system.

• NodeJS: Tooling suite with various Javascript tools.

• NPM: Package manager for the NodeJS Javascript applications.

• Putty: A free and open-source terminal emulator, serial console and network file transfer application.

• Vim: Source code editor.

• Apache Maven: Software project management tool.

• GIT & GitHub: Source code management.

Testing &QA

• Apache JMeter: Application designed to load test functional behaviour and measure performance.

• Cucumber: automated acceptance testing framework.

• Gherkin: DSL for acceptence testing framework.

• Sonarqube: Quality management platform.

• SoapUI: Functional testing tool for testing web services.

• JUnit: Unit testing.

• Mockito: A Mock framework for Unit testing.

The following table presents an overview of the components and the most important technical choices per component.

This chapter gives a more technical overview. It describes each layer of the platform by giving an overview of its packages and the code it contains. Furthermore it describes how a message proceeds through the platform. If you are planning on adding your own Domain Adapter or Protocol Adapter, it will be useful to read this chapter to get a feeling of how the Platform has been built.

A Request through the Platform

The picture below depicts an example of a request (OSLP SetLight request) proceeding through the platform to a device.

• A web request enters the platform at its EndPoint, which in turns calls the RequestService. The RequestService checks if the organisation in the request is authorized, creates the request message and sends it to the MessageSender which in turn puts it on the queue of the Domain Adapter.

• In the Domain Adapter, the incoming message is processed in the MessageProcessor, which in turn calls the Request Service. Here the message is converted to a DTO object. The CoreRequestMessageSender puts the message on the Core Queue.

• The MessageListener in Core receives the message. The DeviceRequestMessageService contains generic functionality such as Authorization, Validation, etc. Once these procedures are completed, the message is routed to the appropriate protocol adapter.

• In the Protocol Adapter the message is received by the MessageListener. It is processed through the MessageProcessor and OslpDeviceService. The request eventually ends up in the OslpChannelHandler, where the actual Protocol Request to the device is made.

For a detailed description of each layer, please take a look at a more detailed description of each layer in this chapter.

Configuration files

The Platform uses property files for certain settings (such as JMS settings, Persistence settings, etc.). These files are stored in property files which can be found in the Config repository on Github. These files are sym linked to /etc/osp/, where the Platform (through reference in context.xml) looks for the property files.

The open smart grid platform use-cases are strongly related to the open smart grid platform domains. Up-to-date information on use-cases can be found on the Grid eXchange Fabric website.

To install the platform you can use one of the following procedures.

1. The Vagrant Installation. This procedure creates and installs a complete image with the Open Smart Grid Platform pre-installed, including all the tools such as Maven, Eclipse, SoapUI, etc.

2. The Manual Installation. Follow this guide if you want to install the Open Smart Grid Platform yourself.

To get started quickly, a Vagrant Installation Guide has been created and a guide for Manual Installation.

The goal of the installation manual is to control a simulated OSLP device through the Platform. Below, is a summary of all steps involved. See the next chapters for a detailed guide with screenshots. Please follow the steps carefully.

A summary of the steps involved:

• Creating a virtual machine using Vagrant and Virtual Box

• Run the puppet script (part of the Vagrant installation), or complete the steps manually (Manual installation)

• Importing Maven Projects into Eclipse

• Creating an Apache Tomcat9 Server

• Setting Up Apache Tomcat9 Server Context

• Deploying all open smart grid platform components to an Apache Tomcat9 Server

• Starting Apache ActiveMQ

• Starting Apache Tomcat9 Server

• Creating the 'test-org' organization in the Database

• Setting up SoapUI

• First SOAP Requests to add a device to the open smart grid platform

• Opening Device Simulator to add a device

• Registering a device

• Using 'SetLight' SOAP Request to switch the light on

Sys Admins who are tasked with keeping the Open Source Grid Platform running on a environment, can find some information in this chapter.

Get Started

To get started with Open Source Grid Platform, please read our . The Introduction offers an excellent overview of the Platform and its features.

A next step could be to have a look at the WSDL's to understand which functions are present per functional domain. Depending on the functional domain one is interested in, one could also have a look at the Protocol Adapter and device simulator for the domain.

Installation

If a full installation is desired, have a look at our . This can be used to setup a development environment which can be used to start the Platform and run it. Installation on one or several servers can be derived from the steps within the Installation Script.

Installation

This document describes the automatic installation procedure for your Open Smart Grid Platform development environment.

Manual installation

If you would like to follow the manual installation procedure, please proceed to the .

Overview

Creating a Virtual Machine using Virtual Box and Vagrant

To improve the usability of the Installation process, a Vagrant file and some puppet scripts are used to automatically set-up an virtual Open Smart Grid Platform development environment. The following steps will describe how to install VirtualBox, Vagrant and kick off the procedure by running the vagrant up command.

System Requirements

The following system requirements are recommended:

• Core i5/i7 ~2.5GHz Dual Core, Quad Core recommended

• At least 16 GB RAM, 32 GB RAM recommended

• At least 20 GB free space, 50 GB free space recommended

The installation procedure has been tested on Windows 10.

Install Vagrant and VirtualBox

note: If you already have VirtualBox, make sure it is at least version 6.1.38

note: Check whether Virtualbox stores the images on a drive with enough free space. (Open Oracle VM VirtualBox Manager -> Preferences -> General -> Default Machine Folder).

note: If you already have Vagrant, make sure it is at least version 2.3.0 Complete the installation and restart your PC.

note: If you did a fresh install of Vagrant and already had a command prompt open, make sure you close this command prompt and open it again.

Tip

• Remember to enable Virtualization in your system BIOS.

• Also disable Hyper-V in Windows (can be found in Windows Features)

Download and run the Vagrant file

First create a new directory (for example: D:\My Vagrant Images\OSGP Development\)

Note

• Make sure that the file is named like this: Vagrantfile without an extension!

• If the file has an extension (for example .txt) you can rename the file using the following console command.

  MacOS/Linux:

  Windows:

Now open a Command Prompt and navigate to the newly created directory where you just put the files. Make sure that you run the Command Prompt using administrator rights.

note: When you open the Vagrantfile you see that default the image is configured to run in virtualbox with 2 cpu cores and 8192 MB of RAM. If you need to you can change this to more or less cpu cores and RAM, but it is recommended to use the provided settings.

Run the following command: vagrant up

note: In case of error bad uri Images/OSGP Development/hashicorp/itc/itc-ubuntu-mate-20.04-amd64 then use the following command;

• vagrant destroy

• vagrant box add itc/itc-ubuntu-mate-20.04-amd64

• vagrant up

note: In case of an error complaining about not being able to resolve a URL (for instance to github.com) then try using a different internet connection not behind a proxy.

Vagrant will now automatically download an Ubuntu image (+- 2.6 Gb), create a virtualbox image from it and run the installation puppet script when finished. This might take a while, depending on your internet speed. After some time (while the script is still running) you will notice that a window with an Ubuntu Virtual Machine pops-up. Don't log in yet, wait until the script in the Console is finished.

Tip

• If the script fails for some reason (eg. Errors in the console such as time outs during downloading), you can retry the procedure by running the following command vagrant destroy && vagrant up

Now that the script has ran its course, it will automatically log in on the Ubuntu virtual machine as user dev.

note: For some actions, like sudo, you will have to enter the password of user dev. The password for user dev is dev.

Optimize your Open Smart Grid Platform Development virtualbox image.

At this point you also can adjust the virtualbox settings like cpus and memory size. If you don't want to adjust this proceed to Chapter 2.1.2. Platform Setup.

If you do want to update the virtualbox settings for this image, shut down the image first:

Once the machine has been Shut Down, open VirtualBox and right click on the new virtual machine (called "OSGP Development") and select Settings. Go to System and increase the Base Memory of the system to at least 6144 MB (6 GB) (or the maximum recommended (in green) amount for your system).

Now go to the Processor Tab and increase the amount of Processors to the maximum recommended (in green) amount.

Close the Settings window and Start the Virtual Machine again. Once it is booted, you should be automatically logged in as the 'dev' user.

Post actions

In order to use git correctly you need to execute the following commands in a terminal:

Recap

Throttling is used to limit the flow into communication channels that need it. The GXF platform has throttling support for CDMA and SMS/texting.

CDMA throttling

The CDMA network uses base transceiver stations (BTS) with cells for each BTS.

Throttling occurrs on the number of concurrent connections per BTS/cell. GXF offers a throttling service that hands out permits per BTS/cell while maintaining the configurable limit.

An application will typically use this service like this:

SMS/texting throttling

SMS can be used to wake up certain types of devices. Throttling in this context means limiting the number of messages per second to a configurable rate.

Basic configuration

In order to use git correctly you need to execute the following commands in a terminal if you haven't done so already:

git config --global user.name "your full name"
git config --global user.email yourEmailAddress

Contributors License Agreement (CLA)

GXF is covered by LFE's CLA. You have to sign/agree to it in order to commit changes. Most likely you will have to sign the individual agreement, unless you work for a licensed company.

Your pull request will detect if you haven't signed it yet and instructions are given on how to continue.

Developer Certificate of Origin (DCO)

GXF uses GitHub's DCO application, so you need to sign-off your commits.

Your pull request will detect if you haven't signed-off your commits and instructions are given on how to continue.

Command line

Just add -s to git commit ....

Note this is the lower case -s (short for --signoff).

With git merge ... you can add --signoff as the shorter -s is used to set the merge strategy.

Eclipse

IntelliJ

There are two procedures for testing the Open Smart Grid Platform.

1. SoapUI. Create and send Soap requests to the Platform to manage a (simulated) light.

2. PublicLighting Demo App. Use the Demo App to send requests to the Platform to manage a (simulated) device.

Setting Up the Open Smart Grid Platform Development environment

This chapter describes all the steps needed to finalize the open smart grid platform development environment.

Lombok

The platform uses Lombok annotations to generate extra Java methods. Without Lombok the project is not imported correctly by Maven and will not run. If you used the Vagrant installation method it should already be installed. To check if Lombok is properly installed to go Help > About Eclipse IDE and scroll down. Here you will see: Lombok <version> "<version name>" is installed. https://projectlombok.org/. If not follow this to install Lombok.

Importing Maven Projects into Eclipse

Open Eclipse by clicking the shortcut on the Desktop and import the projects.

Go to File -> Import -> Existing Maven Projects, browse to folder /home/dev/Sources/OSGP

Import the projects from location /home/dev/Sources/OSGP/open-smart-grid-platform.

Creating an Apache Tomcat Server

In Eclipse go to Window -> Open Perspective -> Debug

In the 'Debug' perspective, go to the 'Servers' view and add a new Apache Tomcat server, Tomcat is available in the folder /home/dev/Tools/tomcat (or in another location if you didn't set up a VM using Vagrant, the latest version usually works fine).

Click on Next

Click on Finish

Make sure that Tomcat is using the correct Java Runtime Environment:

Click on 'Runtime Environment', 'Installed JREs' and click 'Add'. Choose 'next' with 'Standard VM' highlighted. Click 'Directory' and in the folder browser that is shown, use the key combination 'Ctrl + h' to show hidden files. Choose '.sdkman/candidates/java/current' and select 'Open' and 'Finish'. In the 'Installed JREs' screen, unselect the default JVM (which will be an Eclipse hotspot JVM) and select '/home/dev/.sdkman/candidates/java/current' then close the screen by clicking 'Apply and Close'. The last step is to choose the new JRE which was just added in the dropdown of the 'Runtime Environment' screen by selecting '17.0.5'. Click 'Finish' to apply.

Setting Up Apache Tomcat Server Context

All modules contain their own context.xml. In the module specific context.xml are the environment variables defined where the global and module specific configuration files are located. Default they will point to a location in /etc/osp/.

Deploying all Open Smart Grid Platform components to Apache Tomcat Server

Starting Apache ActiveMQ

Continue with starting Apache ActiveMQ. If you installed an environment as described with Vagrant, you can double click the ActiveMQ shortcut on the desktop.

Alternatively you can open a terminal and run the executable manually by using the following command: (the executable can be found in the folder /home/dev/Tools/activemq/bin)

Starting Apache Tomcat Server

With ActiveMQ running, the Tomcat server can be started. Go to Eclipse, go to the Servers tab in the Debug view, and right click on the Tomcat server and select 'Start'.

note: In case of an error starting up for the very first time, try and start up only the module: 'osgp.core' first. This makes sure the database scripts are executed.

Probe

Starting pgAdmin III and Connect to PostgreSQL

Open pgAdminIII and configure a connection: choose the 'Add a connection to a server.' and fill out the fields using

• Host: localhost

• Port: 5432

• Username: osp_admin

• Password: 1234

Creating the 'test-org' Organization (in database osgp-core)

Run the script in /home/dev/Sources/OSGP/Config/sql/create-test-org.sql to insert 'test-org' organization into the organisation table of the osgp_core database.

If asked for a password, enter 1234

Go back to PgAdmin III, expand servers, select localhost -> databases -> osgp_core -> Schemas -> public -> Tables. Right click the organisation table and select to view data for the top 100 rows. Confirm that the test-org organisation has been added to the Database.

Now that everything has been set up, continue to the next chapter to start testing the Platform by sending it some requests.

Manual Installation

This chapter describes the steps for a manual installation (eg. not using the vagrant script and puppet scripts). This chapter is for developers who would like to have more control over the installation procedure.

With the increased control come increased risks of things not working with the versions or configuration of the software involved with the OSGP environment. If you run into issues, you may find clues in the puppet scripts about versions and modifications to the configuration of installed software.

Note

• Skip this chapter if you followed the Vagrant installation! You can continue with next chapter: Setup the Open Smart grid Platform

Operating System

The Open Smart Grid Platform runs on a Linux environment. It is recommended to set up a machine running Ubuntu.

Software and tools

The Open Smart Grid platform needs the following software and tools installed/downloaded:

• Java 8 openjdk-8

• PostgreSQL and pgAdmin 3

• Git

• Maven

• ActiveMQ

• Tomcat

• Apache HTTPD

• SoapUi

• Eclipse IDE for Java EE Developers

• Google Protocol Buffers: protobuf-compiler, libprotoc7 and libprotobuf7

• PostgreSQL JDBC driver

Settings

User

It is recommended to create a 'dev' user, because some scripts contain hard coded references to this 'dev' user. It is possible to skip this step, but then some of the scripts will have to be adjusted manually.

Tomcat

• Place the PostgreSQL JDBC driver jar in the Tomcat lib directory.

• Change permissions of Tomcat Config files to 644 in the Tomcat conf directory.

Apache HTTPD

• Enable mod_ssl by running the following command:

  Copy

  a2enmod ssl

• Enable proxy_ajp by running the following command:

  Copy

  a2enmod proxy_ajp

Java

• Make sure the JAVA_HOME var is set, and points to openjdk-8.

Cloning Sources

Clone the following repo's, it is recommended to create a Sources/OSGP directory in /home/dev/ since some scripts contain hard coded references to those folders.

git clone https://github.com/OSGP/Config.git /home/dev/Sources/OSGP/Config
git clone https://github.com/OSGP/open-smart-grid-platform.git /home/dev/Sources/OSGP/open-smart-grid-platform
git clone https://github.com/OSGP/Documentation.git /home/dev/Sources/OSGP/Documentation

Make sure you are on the development branch (default).

Creating directories and symlinks

Create the following directories:

• /var/log/osp/logs

• /etc/osp/

Make the dev user (or equivalent) the owner of the log directory with rwx permission. Give the other users read and execute permission.

Execute the script /home/dev/Sources/OSGP/Config/scripts/create-symlinks.sh

Note This script uses hard coded references to /home/dev/Sources/OSGP/*, if you used a different user, please edit the script before executing it.

The script will make symlinks to certificates, to Apache HTTP server configuration and copy configuration settings as samples to locations where these properties may be overridden.

Initiating the Database

To create the database run the following command (Change /home/dev/ in case of no dev user)

sudo -u postgres /usr/bin/psql -p 5432 -f /home/dev/Sources/OSGP/Config/sql/create-users-and-databases.sql

And create a backup of the pg_hba.conf file (modify if your version of PostgreSQL is different)

cp -p /etc/postgresql/9.4/main/pg_hba.conf /etc/postgresql/9.4/main/pg_hba.backup

Finally, reload the postgresql service:

service postgresql reload

Set up the Open Smart Grid Platform

Continue with setting up the Open Smart Grid Platform by following the Set up the Open Smart Grid Platform Guide

Testing the platform

This chapter will describe the steps needed to test the Open Smart Grid Platform.

Setting Up SoapUI

Start SoapUI by double clicking the shortcut on the Desktop or run it manually by typing the following command in a terminal:

Go to File -> Preferences -> SSL Settings, and browse for the KeyStore to /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/test-org.pfx and fill out the password (the password is 1234)

Go to WSDL Settings and check 'Generate Example Values in New Requests' and 'Generate Comments with Type Information in New Requests'

Add the SoapUI projects to SoapUI

There are several SoapUI project prepared, see /home/dev/Sources/OSGP/Config/soapui/. Import all SoapUI projects present in the folder mentioned above. Below, 2 projects are shown as examples.

Adding the 'Admin' Soap project

Import the 'admin' project by clicking File -> Import project. Browse to /home/dev/Sources/OSGP/Config/soapui/, select 'admin-soapui-project.xml' and click open.

Alternatively you can create the 'admin' project yourself by following the steps below:

• Create a new SOAP Project and call it 'admin'

• Open the Project View by double-clicking on the 'admin' project. Go to 'WS-Security Configurations' and select the 'Keystores' Tab. Click on the '+' to add the test-org.pfx in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

• Fill out the password (1234) and click Ok and close the Project View window.

• Right click the 'admin' project and choose 'Add WSDL'. Enter the following URL in the WSDL Location field:

• Make sure the box 'Create sample requests for all operations' is checked, and click OK.

Adding the 'Public Lighting' Soap project.

Import the 'public-lighting' project by clicking File -> Import project. Browse to /home/dev/Sources/OSGP/Config/soapui/, select 'public-lighting-soapui-project.xml' and click open.

Alternatively you can create the 'public-lighting' project yourself by following the steps below:

• Open the Project View by double-clicking on the 'public-lighting' project. Go to 'WS-Security Configurations' and select the 'Keystores' Tab. Click on the '+' to add the test-org.pfx in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

• Right click the 'public-lighting' project and choose 'Add WSDL'. Enter the following URL in the WSDL Location field:

First SOAP requests to add a device to the open smart grid platform

Before sending the request, the test-org.pfx should be added as SSL Keystore: Go to the properties interface for the request (bottom left of the screen, after selecting 'Request 1' under UpdateKey in the 'admin' project'), and choose test-org.pfx from the drop-down box.

Note

• This has to be done for each request!

An SSLD needs to be added to the platform, as well as a manufacturer and a public key for the SSLD. A couple of steps need to be performed to realize this.

1 Add manufacturer 2 Add device model 3 Add SSLD 4 Setup a protocol for the SSLD to use 5 Set the public key for the SSLD (in case of OSLP)

The AddManufacturer function adds a new manufacturer to OSGP. All devices are coupled to a manufacturer.

The AddDeviceModel function adds a new device model to OSGP. All devices are coupled to a device model.

The AddDevice function adds a new SSLD to OSGP. The device is coupled to a device model and a manufacturer.

The function UpdateDeviceProtocol sets a protocol for a device.

The UpdateKey function of the admin webservice sets a public key for a device. Double click 'Request 1' under UpdateKey in the 'admin' project. Add the following request:

Opening Device Simulator to Add a Device

In order to be able to use the SSLD-000-00-01 Device, the device needs to be simulated in the Device Simulator. To do this we have to create it. In the Firefox Browser, open the Device Simulator by going to the following URL:

Fill out the fields like this:

• Device Identification: SSLD_000-00-01

• IP Address: 127.0.0.1

• Device Type: SSLD

• Protocol: OSLP ELSTER

Registering a Device

Using 'SetLight' SOAP Request to Switch the Light On

Now that the Device is known in the platform, and simulated in the Device-Simulator, the device can be used. Let's switch on the Light. Using SoapUI, click on Request 1 under SetLight at the public-lighting project. Set the following parameters in the request (And do not forget to set the Keystore in the request properties):

The last request concerns the response from the previous SetLight request. In SoapUi open Request 1 under 'GetSetLightResponse' in the 'public-lighting' project. Set the following parameters in the request (And the keystore in the request properties). Make sure to replace the CorrelationUid with the value from the respons from the SetLight request.

Note

• Do not forget to set the CorrelationUid to value in the response you received from the setLight request.

The server replied Ok, indicicating that the SetLight request has been processed succesfully.

This step also concludes the installation manual.

Testing the platform using the Web Demo

This chapter will describe the steps needed to test the Open Smart Grid Platform using the Web Demo Application.

If you followed the steps from the 'Platform Setup' then you can skip the following installation steps, because the Web Demo will already be available.

Installing the Web Demo

• Open Eclipse and import the following folder as Maven project /home/dev/Sources/OSGP/open-smart-grid-platform/public-lighting-demo-app/

• Add the PublicLightingDemoApp components to the Tomcat Server.

• (Re)Start the Tomcat Server.

Creating a device

To access the Demo App go to the following URL: https://localhost/web-demo-app/

If you encounter an Untrusted Connection page, go to 'I Understand the Risks' -> Add Exception.. -> Confirm Security Exception

Click the Add a Device button in the Menu bar, and enter SSLD_000-00-01 at the Device Identification field and press Submit.

The following screen will appear, it shows that the device has been successfully added to the Platform.

Opening Device Simulator to Add a Device

In order to be able to use the SSLD-000-00-01 Device, the device needs to be simulated in the Device Simulator. To do this we have to create it. In the Firefox Browser, open the Device Simulator by going to the following URL:

Fill out the fields like this:

• Device Identification: SSLD_000-00-01

• IP Address: 127.0.0.1

• Device Type: SSLD

• Protocol: OSLP_ELSTER

You should return to the Devices screen and see the message "Device with identification SSLD_000-00-01 was created."

Registering a Device

Then click the 'Confirm Registration' button. The message should read: "Device with identification SSLD_000-00-01 was confirmed to be registered."

Using 'SetLight' Switch the Light On

Now that the Device is known in the platform, and simulated in the Device-Simulator, the device can be used. Let's switch on the Light. Go to the Demo App and press the List button in the Menu. A list should appear, showing the device that has just been added using the Add Device button.

Click on the Manage button to access the Device Details.

Switch on the Light by setting the Light Value to 100 and by checking the 'LightOn' checkbox (as shown in the screenshot below)

Hit submit to submit the request to the Platform. The following screen should appear:

In the home screen of the OSLP device simulator, the lightbulb should light up for SSLD_000-00-01. This means that the request succeeded.

This step also concludes the installation manual.

We have prepared a full set of configuration (properties files, certificates, key store, ...) which is available in our Config repository. This configuration can be used to setup a trial environment.

Soap Requests in GXF have fields for an User name and an Application name in the Header. These are only used for logging and auditing. There are no restrictions (e.g. Authorizations) with regard to these fields. However, it is recommended to use names for Logging and Auditing purposes.

Platform uses single device calls.

In order to add a device to the platform, call the add device method in the device installation web service (for a specific domain) for each device.

Implementation may differ for different device types

Each device type may require its own set of attributes, security settings, etc. Each device type may have its own registration mechanism.

• To add a DLMS Device to the Platform, take a look at the documentation of the Soap call AddDevice as defined in the SmartMeteringInstallation.wsdl.

• To add an OSLP Device to the Platform, the Soap call defined in CoreDeviceInstallation can be used, or the UpdateKey Request.

Please take a look at the chapter Testing the open smart grid platform in the installation manual for a detailed guide of how to add a OSLP device to the platform.

Additional Device actions

In the Domain Chapter of the documentation more information of the Web Service calls can be found for Adding devices, setting configuration, authorizations, schedules, firmware updates, etc.

Adding an Organisation to the platform

This chapter describes how to add a new Organisation to the platform. This includes creating a certificate for the new organisation.

Creating an Organisation

In SoapUi go to DeviceManagement under the Admin project. Click on request 1 under CreateOrganisation. Fill in the parameters like the example request below. Make sure to sign the request with the test-org.pfx and to use the test-org organisation in the request header.

This request creates a new organisation called "my-org":

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:CreateOrganisationRequest>
         <ns1:Organisation>
            <!--type: Identification-->
            <ns1:OrganisationIdentification>my-org</ns1:OrganisationIdentification>
            <!--type: string-->
            <ns1:Name>my-org</ns1:Name>
            <!--type: string-->
            <ns1:Prefix>MYO</ns1:Prefix>
            <!--type: PlatformFunctionGroup - enumeration: [ADMIN,USER]-->
            <ns1:FunctionGroup>ADMIN</ns1:FunctionGroup>
            <!--type: boolean-->
            <ns1:Enabled>true</ns1:Enabled>
            <!--1 or more repetitions:-->
            <!--type: PlatformDomain - enumeration: [COMMON,PUBLIC_LIGHTING,TARIFF_SWITCHING]-->
            <ns1:Domains>COMMON</ns1:Domains>
            <ns1:Domains>PUBLIC_LIGHTING</ns1:Domains>
         </ns1:Organisation>
      </ns1:CreateOrganisationRequest>
   </soapenv:Body>
</soapenv:Envelope>

Authorise the new Organisation for the device

To use this new organisation to control the SSLD_000-00-01 device, the authorisations need to be updated. To do that the UpdateDeviceAuthorisations request will be used, it can be found under DeviceManagement in the admin project.

Fill in the parameters like shown below. The functionGroup will be set to AdHoc to authorise the 'my-org' organisation for the AdHoc functions.

Make sure to use the test-org as OrganisationIdentification in the request header, and to sign the request with the test-org.pfx.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>AAPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:UpdateDeviceAuthorisationsRequest>
         <!--1 or more repetitions:-->
         <ns1:DeviceAuthorisations>
            <!--type: Identification-->
            <ns1:deviceIdentification>SSLD_000-00-01</ns1:deviceIdentification>
            <!--type: Identification-->
            <ns1:organisationIdentification>my-org</ns1:organisationIdentification>
            <!--type: DeviceFunctionGroup - enumeration: [OWNER,INSTALLATION,AD_HOC,MANAGEMENT,FIRMWARE,SCHEDULING,TARIFF_SCHEDULING,CONFIGURATION,MONITORING]-->
            <ns1:functionGroup>AD_HOC</ns1:functionGroup>
            <!--Optional:-->
            <!--type: boolean-->
         </ns1:DeviceAuthorisations>
      </ns1:UpdateDeviceAuthorisationsRequest>
   </soapenv:Body>
</soapenv:Envelope>

Creating a certificate for the new organisation

Now that the 'my-org' organisation is authorised to use the SSLD_000-00-01 device, it is time to create a certificate for the my-org organisation. This certificate will be used to sign the requests.

Open a terminal and navigate to /home/dev/Sources/OSGP/Config/certificates/

A script has been created to create the certificates, execute it by running the following command in the terminal:

./create_client_cert.sh my-org 1234 1234

You should receive similar output as shown in the screenshot below.

Now that the certificate has been created, restart the tomcat server.

Signing a request with the new certificate

When the tomcat server is up and running again, go to SoapUi and add the new certificate to the public-lighting project: double-click on the project, go to the WS-Security Configurations tab and select the keystores tab. Click the '+' button and browse to the my-org.pfx certificate which can be found in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

Now double-click on 'Request 1' in SetLight in PublicLightingAdHocManagement in the public-lighting project. Set the SSL Keystore to 'my-org.pfx' in the request properties so the request gets signed with the new certificate. Change the request parameters as shown in the example below:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>my-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:SetLightRequest>
         <!--type: Identification-->
         <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
         <!--1 to 6 repetitions:-->
         <ns1:LightValue>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:Index>0</ns1:Index>
            <!--type: boolean-->
            <ns1:On>true</ns1:On>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:DimValue>50</ns1:DimValue>
         </ns1:LightValue>
      </ns1:SetLightRequest>
   </soapenv:Body>
</soapenv:Envelope>

Note the OrganisationIdentification is now set to 'my-org'. Send the new request, you should receive the following response:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Header/>
   <SOAP-ENV:Body>
      <ns2:SetLightAsyncResponse xmlns:ns2="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10" xmlns:ns3="http://www.opensmartgridplatform.org/schemas/common/2014/10">
         <ns2:AsyncResponse>
            <ns3:CorrelationUid>my-org|||SSLD_000-00-01|||20160805150420802</ns3:CorrelationUid>
            <ns3:DeviceId>SSLD_000-00-01</ns3:DeviceId>
         </ns2:AsyncResponse>
      </ns2:SetLightAsyncResponse>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Check the device-simulator to see if the dimValue of the SSLD_000-00-01 changed to 50.

You now have successfully created a new organisation, along with a certificate to sign the requests, and changed the device authorisations of the device to accept commands from the new organisation.

CDMA throttling

The main config is enabling the throttling client in your application

The throttling client needs to know where to find the throttling service and tell the service what threshold to use. This is configured using the following settings:

Multiple clients can use the same configuration by using the same throttling.configuration.name. Throttling will be globally handled across those clients. Make sure all clients that use the same name also use the same throttling.configuration.max.concurrency. The last client to register a configuration will determine the max concurrency (overwriting previous registrations).

The throttling service itself should be deployed only once, it is not suited for load balanced deployment.

All the features of the open smart grid platform are accessible by it's webservices, as defined in the WSDL files. To use these services to communicate with devices through the platform, please keep in mind the following things.

• A Soap request will have to be generated from your Application.

• In the WSDL and xsd files of the Platform the Requests and it's objects are defined.

• Fill in the parameters once you have an empty soap request for a certain function. The restrictions/requirements are defined in the WSDL files.

• The request should have a header where an ApplicationName, UserName and OrganisationId are defined. At the moment User and App name are not used in the Platform (Except for audability and logging). The OrganisationId, however, must match a known organisation within the Platform. This organisation must have the right authortities to make a certain request.

• Furthermore, the Request must be signed with a certificate from the Organization with the OrganisationId in the header.

• The requests use the Secure HTTP protocol (https).

• For Asynchronious requests, the Platform will respond with a correlationId after succesfully making the request. Use this correlationId in the matching Response-Request to obtain the response from the device.

To learn more about the open smart grid platform's webservices, please take a look at the or for hands-on experience with the Platform's webservices follow the .

Rest

It is not possible to communicate with the Platform directly using REST webservices. As mentioned above, the open smart grid platform uses SOAP (For reasons defined ). If you want to use REST for your front end, you can write an Integration Layer that serves as a Soap Client and exposes the Soap calls through Rest web services that you can access from your front end.

Flows

When using the SOAP Web service, there are 2 flows that can occur:

• some calls are synchronous: a response is returned immediately;

• other calls are a-synchronous: an initial response contains a correlation id, which can be used to obtain the actual response.

We invite everyone to participate in the development of the open smart grid platform. There are multiple ways to support the project.

• submit issues about bugs or requested features

• solve issues

• develop new features

• write or improve the documentation

This chapter contains all the non-technical and general technical knowledge for developers to start contributing.

• How to start everything up? Make sure that PostgreSQL is running. Make sure that Apache HTTPD web server and Apache ActiveMQ are running. Then start Apache Tomcat application server as described in the installation manual.

• Where are the log files? The Apache Tomcat application server logs can be found in /var/log/tomcat. The Apache HTTPD web server logs can be found in /var/log/httpd. The PostgreSQL log files can be found in /var/lib/pgsql/9.3/data/pg_log. The platform log files can be found in /var/log/osp/logs/.

• Help: SELinux is preventing < something >? Make sure to set SELinux to 'permissive' mode. Then try again and it should work as SELinux will no longer enforce the current policy. Later, one can use the SELinux tools to create a proper policy that allows everything that was prevented before.

• How to add or update a component? Make sure to place the properties file(s) for the component in /etc/osp. Add the locations of the properties file(s) to Apache Tomcat context.xml file. Add the war file to /usr/share/tomcat/webapps. Restart Apache Tomcat.

• How to configure a component? Most (if not all) components of the open smart grid platform are de-coupled using queues. Configure the broker URL in the properties file and take notice of the queues that a component uses/needs. Make sure to double check the connectionstring for PostgreSQL.

• How to configure URL's for a component? In this case the Apache HTTPD vhost needs to be updated. The vhost config file can be found in /etc/httpd/conf.d. We use redirects from HTTP to HTTPS and AJP proxy to send the requests to Apache Tomcat.

• How to check up on Apache ActiveMQ? Apache ActiveMQ offers a web interface (the default port is 8161, default credentials: admin/admin). Using the web interface one can check the queues and especially the dead letter queue (DLQ).

• How do I obtain a public key for a device? Public Keys are usually manufacterer supplied for Smart Metering. For the Open Source part you can use the OSLP device simulator. In Sources/OSGP/Config/certificates/oslp/ you can find instructions for generating a OSLP device public key, and a folder with 5 pre-generated keys for test devices.

• My code gives a lot of errors in Eclipse after importing Try the following things: run mvn clean install in the open-smart-grid-platform directory. Right-click on a project in Eclipse and select 'Maven -> Update project..', select all projects and update.

• The Vagrant script fails If you are receiving errors while downloading the ubuntu iso, sources, etc. or if the puppet script does not start; try running the Vagrant script again by typing vagrant destroy && vagrant up in the directory with the vagrant file.

• I want to update my code from Github If you want to update your code, just run git pull in the repository you want to update. You can also create a fresh Virtual Machine using the vagrant installation, this procedure makes sure you have the most recent code.

• Is a user required to consume platform services? No, an organization is required.

_If your question is not in this list, please create an [issue on Github in the documentation repository ]documentation repository

Deployment options

Hosting the open smart grid platform in the cloud is possible, as well as on premises.

Active-active setup over multiple datacenters

If you like to setup an active-active installation over multiple datacenters. Make sure that the open smart grid platform database runs redundant over both datacenters (master-slave configuration).

Maintenance

There's not much maintenance that needs to be performed. Archiving some old log files, checking up on available disk space and creating a backup of the databases. Looking into the queues to see if there are no messages in the dead letter queue.

This is a guide to start contributing to the open smart grid platform project:

1. Make yourself comfortable with the open smart grid platform by e.g. installing it.

2. Read the documentation to get an idea of how the software works (architecture chapter), how the community works (this chapter).

3. Find an open issue to work on or fire an new issue.

4. Assign yourself to the issue and start contributing.

5. Start contributing by using the procedures mentioned in this chapter.

Thank you for contributing to the Open Smart Grid Platform. Please keep the following in mind before submitting code.

Guidelines

Before code is merged it needs to comply with a number of guidelines: 1. Code should be as complete as possible (preferably no placeholders, TODO's or FIXME's) 2. Right formatting; code should follow the Coding Conventions (see 3.1.2) 3. Fixed/added unit tests where applicable 4. Javadocs added where applicable 5. Accepting pull-requests with SonarQube reports "Blocker" and "Critical" are not allowed 6. Comply with International open standards where possible (e.g. IEC standards)

Contributor License Agreement

We ask each of our contributors to sign our contributor license agreement (CLA). This has advantages for both parties, it gives you the assurance that your contribution will remain available under the Apache 2.0 license. Meanwhile, you give your code in license to us, so we can add your code to the open smart grid platform. And we know your contribution is entirely your work, so we don't get in trouble with legal issues. Please read the CLA text carefully.

Submitting code

To submit changes to the open smart grid platform branches: 1. Create a fork of the open smart grid platform repo you will be working in 2. Make and commit your changes to your fork 3. Create a pull request to merge the changes into the right branch (see 4.1.4 for the branching strategy) If the changes fix a bug, mention the issue number in the commit message or pull request message (example: fixes 101, solved 87). If no ticket exists, create one beforehand. Afterwards, please update the relevant documentation in this GitBook.

Open Source Branching Strategy

The open smart grid platform's main branch is master. All major releases are tagged in this branch. Development is done in the development and feature branches. We use the GitFlow branching strategy. Find more information on this strategy here: GitFlow

The GitFlow workflow is someone complicated, but it has the advantage that it gives a clear overview of all previous releases and current development and thus helps to collaborate more efficiently. Please follow this strategy in your commits.

Pull requests: review process

Anyone can send in a pull request. Assign a maintainer or other developer with knowledge on this topic to accept/evaluate your pull request. You can view the SonarCube test results at (http://ci.opensmartgridplatform.org/sonarqube/) and the Jenkins continuous integration results at (http://ci.opensmartgridplatform.org)

If your code is a useful contribution and meets our quality standards (see section 3.1), it will be added to the open smart grid platform! Developers are in charge of judging this. Don't forget to update the documentation as well.

Developer tools, technical guidelines and continuous integration

Tools Used

The technology and tools used can be found in the Technology stack section.

Code Guidelines and Code Tests

• To prevent formatting issues caused by using different IDEs, the formatting style for the code is based on Google Java Format with some extra additions based on the Eclipse based conventions that were already in use.

  Check the explanation of the general ideas as well as the instructions how to configure your idea to be able to format according to the conventions (available for Eclipse and IntelliJ).

• Follow GitFlow approach for branching

• Write behaviour driven tests using Cucumber and Gherkin, see the Integration-Tests

• In case you are not familiar with behaviour driven tests, include unit tests

• Fix SonarQube issues

• Issue pull request (preferable to development branch)

Continuous Integration

All changes pushed to GitHub are built by our build server. Pull requests to master branch or development branch are also built. SonarQube performs static code analysis to help improve the quality level of the code base.

• Jenkins buildserver: An open source continuous integration tool written in Java.

• SonarQube: SonarQube is an open platform to manage code quality.

Technical Conventions and Rules for New Code

This project is engineered, built and tested using Domain Driven Design and Behaviour Driven Tests.

• Use the Frameworks, don't roll your own

• Single class, single responsibility

• Value objects are immutable

• Map generated objects to value objects or entities

• Transferring an object means using a DTO

• Use base classes for common logic

• Interfaces are good, but 'impl' is bad

• Extend classes, don't expand classes

• Migrate databases using Flyway

• JMS for messaging

• Extend the authorization logic if needed, don't bypass it

• Log errors/exceptions

• Add meaningful comments to the code

• Follow the code guidelines

Development Guide per Component

Web Service Adapters

The web service adapters use Spring Web Service, contract first. JAXB is used to generate Java classes from the XSD's. All SOAP operations are bound to an endpoint. The incoming SOAP requests are authenticated by organization identification (plus certificates). Organization authorizations are checked for the desired operation. If the request is OK, a JMS message is sent to a domain adapter component.

Domain Adapters

Domain adapters contain business logic and persistence. Domain adapters process and forward the JMS message to the Core component.

Domain Components

Domain components contain entities and value objects.

Core

The Core component routes messages from domain adapter components to protocol adapter components and vice versa. Furthermore, it offers read-only database access for protocol adapter components.

Protocol Adapters

Protocol Adapter components translate a message from domain adapter components into a protocol message for a smart device. Protocol Adapter components send the protocol message to a smart device using a network connection. The response from the smart device is translated into a domain response message which will be sent to the Core components (which will route it to the domain adapter which issued the request).

OSLP

For the OSLP implementation, two components are used. The first component is the protocol adapter for the protocol. It can translate message into the protocol message for SSLD's. Second there's the signing-server component. It is responsible for signing the protocol message using the private key of the platform. The components communicate using a key-pair. The signing-server can handle multiple protocol adapter instances by utilizing a reply-to queue per protocol adapter instance. Since the protocol adapter component needs to be reachable from a network, it is a requirement that the private key may not be used by the protocol adapter directly. The signing server component can be deployed in such a way that no network access is available to this component, as the only coupling needed are the queues / the message broker.

Other Guides

Installation Guide

If a full installation is desired, have a look at our Installation Guide.

Create New Domain Guide

In order to add a new domain to OSGP, you can benefit from the information in the Create New Domain Guide.

Documentation Publication

This documentation is available in multiple formats.

Web

• Development branch publication

PDF

• Click on Export as PDF on right side of gitbook Development branch publication

Contributing to documentation

The documentation is build using GitBook software from Markdown files in the documentation repository.

We encourage you to participate in improving the documentation. From corrected typos to new sections, every commit is appreciated. You can access the source files by clicking the "Fix this page"-button in the GitBook or by selecting the relevant Markdown-file in the documentation. You can commit your changes by sending a pull request.

1. Fork the repo, do work in a feature branch.

2. Issue a pull request.

3. Make sure the automated test suite succeeds. They will show-up in the pull request.

4. Sign the CLA using EasyCLA.

5. Assign a maintainer or other developer on this topic to accept/evaluate your pull request. The current maintainer can be found in the governance paragraph.

Some information on GitBook and using Markdown can be found here, more elaborate information on GitHub-flavored Markdown is found here. If you like to upload illustration, you can use git or Github

If you are completely new to this and you need help to get started, file an issue in the documentation repository.

Chapters in the documentation

It may be obvious to you already from the index, however, here is an overview on what documentation goes in which chapter.

• Chapter 1 consist of an open smart grid platform introduction and architecture for potential users, architects and developers. The open smart grid platform website is used for basic product information.

• Chapter 2 contains the general userguide for open smart grid platform users

• Chapter 3 contains community related topics

• Chapter 4 contains information related to the open smart grid platform domains

• Chapter 5 contains information related to the protocols and simulators

• Chapter 6 shows the support options

• Chapter 7 code and documentation license