This chapter contains the general architecture and properties of the Open Smart Grid Platform. Domain and protocol specific information can be found in the domain and protocol chapters. This chapter is written for (potential) users, architects and developers.

The Open Smart Grid Platform is designed for message based communication.

• Acts as a connecting link between (web)applications and smart devices

• The open Source approach prevents vendor lock-in

• State of the art security

• Fully scalable, dynamically scaling up and down as more devices and applications are added.

• Freedom of choice in the desired IP communication infrastructure , e.g. CDMA and GPRS.

• Stimulates open innovation by using open standards and open source technology

• Multiple devices and communication protocols are supported

• Independent of (cloud) hosting infrastructure

• By de-linking the chain and the use of open standards and the open source license, anyone can build his or her applications on top of the open smart grid platform.

• The open smart grid platform is optimized to provide reliable and efficient delivery of command and control information for e.g. smart meters, direct load control modules, solar panels, gateways and other applications.

• The open smart grid platform simplifies the implementation of smart devices resulting in a shorter time-to-market by having built-in device management features

• The platform supports various IP data communication infrastructures to communicate with the devices (internet, lan, GPRS, CDMA, UMTS, etc.).

• The open smart grid platform also supports authentication and encryption for all data exchanges to protect the integrity and privacy of data as required in e.g. the smart grid.

• The open smart grid platform supports multiple protocols

• Easy application integration

• Supports active-active setup over multiple data centers

• Adding servers can be done in runtime

Please note: the Open Smart Grid Platform is not built for streaming data such as video, audio or a stream of high frequency measurement data.

Unique features of the Open Smart Grid Platform

The Open Smart Grid Platform is unique due to its multi-dimensional, generic and open design. Because of a true separation of layers and the use of open standards, other suppliers and/or third parties are able to develop and market innovative solutions.

1. The platform is multi-dimensional. This means that several customer use cases (with separate business models) are able to use the various device functions. One single application could use the same function of different devices.

2. The generic design ensures that the platform can be used in a flexible way for several functions and applications (e.g. public lighting services and smart meter services).

3. The platform is aimed at the 'common parts' of the technology chain; suppliers or vendors (of both applications and devices) have no competitive advantage in delivering these kind of services.

4. The platform layers are truly separated by open standards and the platform is made available as open source software.

5. The platform does not store any application data (the platform is thus stateless ). No messages/commands will ever get lost.

   This enables third party vendors and developers to deliver innovative applications which are competitive in both rich functionalities and the generated data.

The non-functional view is an overview of the most significant non-functional demands.

The identified non-functional demands are:

• Time Behavior

• Extensibility

• Internationalisation and localisation

• Security

• Scalability

Authentication of open smart grid platform

The Open Smart Grid Platform contains an extensive authorization model, which enables a device owner to give certain rights on certain devices to other organizations. Every organization will only see devices they have rights to.

This model displays the most important entities of the open smart grid platform system and their mutual relationships.

Image of Logical Authorisation Data Model

The logic of the model above:

At the top of the image is the entity "authorisation". This represents the permissions of an organization on a certain device. In general an organisation will have a lot of permissions, at least one for each device it needs to manage.

The functions an organisation can execute on a device are determined by the function group the authorisation refers to. Function groups are collections of functions and are predefined in the software. The following function groups have been predefined.

• Owner-group (this contains all functions)

• Ad hoc-group (Functions for ad hoc switching of lighting)

• Management-group (Platform functions)

• Installation-group (Functions to install devices)

• Firmware-group (Functions for updating firmware)

• Schedule-group (Functions to create lighting schedules)

• Tariff scheme-group (Functions to create tariff groups)

• Configuration-group (Functions to configure devices)

• Monitoring-group (Functions to monitor devices)

This structure provides maximum flexibility when assigning rights to devices. Devices always belong to an Owner. An owner is an organisation, but not every organisation is an owner. A device can have more than one owner. The entity "Event", at the bottom of the image, is the execution of a function by an organisation on a device.

Details like device-type, device-status, etc. have been omitted from this model.

One security requirement is that each event must be traced back to a 'natural' person, also known as an audit trail. Although the open smart grid platform does not register individual users we can meet this requirement by registering a data-item with each event. This enables the user organisation to investigate which events belongs to which 'natural' person. This data-item can for example be an user-ID provided by the user organisation which doesn't have to be unique in the open smart grid platform.

Table describing the entities in the logical data-model

An organization can get rights to one or more function groups, and thus all functions in that function group will be available to this organization.

To ensure that devices can only receive instructions from a 'genuine' open smart grid platform it must be possible to authenticate the open smart grid platform. This is implemented through a standard technology based on asymmetric encryption (if supported by the Device). The open smart grid platform will receive a unique key to enable the devices to tell if the messages come from a 'genuine' open smart grid platform. Both OSLP and DLMS device types use this kind of encryption. To prevent replay-attacks each message will get an index number (this is standard practice as well).

Authentication of devices

To ensure that the open smart grid platform can distinguish between 'genuine' devices and 'illegal' devices, all devices are supplied with a manufacturer key. Each device has a unique key. Because of the asymmetrical encryption the platform contains the public part of each key. In this way devices can be identified by their unique key and their unique hardware ID. The device-ID will be encrypted in each message sent from the device to the platform.

All communication between the open smart grid platform and the devices will be signed with these keys to ensure (1) the source is legitimate and (2) to ensure the integrity of the message. It is not necessary to encrypt the whole message because confidentiality is not important. This results in a less computationally intensive process.

When a key is stolen (by hacking a device) this will not affect the integrity of the other devices. Each device has an unique key after all and only the hacked device has to be excluded from communication in the platform.

The security is independent from the carrier (GPRS, CDMA, Ethernet, etc.). The open smart grid platform supports symmetric and asymmetric encryption (depends on device and protocol).

For OSLP devices, the firmware will be used to distribute keys to devices. In this way we can use the existing secure firmware update mechanism for updating keys and certificates. DLMS devices use a mechanism to switch keys that is not dependent on firmware updates.

Additional security may be provided by using TLS communication.

Authorisation of organisations

Authorisation for use of the platform functionalities is handled by function groups. Function groups are defined for both platform functionality and device functionality. Each function group has one or more functions. Access to device functions can be set per device. The tables below show an overview of all function-groups and device-functions and platform-groups and platform-functions respectively.

This are some examples how a message flows in the Open Smart Grid Platform.

Message Flow: Request/Acknowledge/Poll

Message Flow: Request/Acknowledge/Notify

*In case the response is not timely retrieved by the client app, OSGP will resend the notification with correlation id to the client app. The amount of retries is configurable.

This chapter describes the possibilities of a redundant set up of the Open Smart Grid Platform. The Platform is designed to run in a High Available (or HA) environment, and to prevent data loss due to unexpected failures. Each component of the Platform is designed to be stateless. Components communicate with each other using message queues, which are processed in an asynchronous way.

Active-active

In an active-active setup, multiple instances of each component (eg. Web Services, Core, Protocol Adapter) process the data at the same time. Traffic is equally distributed across the instances. In case of a defect in one instance the traffic is automatically processed by the remaining instance(s). The Open Smart Grid Platform is designed to run in such a set up, and thus preventing down time in case of a failing server. Each component of the Platform can run in an independent, redundant and scalable way.

Database

The Open Smart Grid Platform uses a PostgreSQL database. PostgreSQL supports multiple database servers. For example, a slave and master node, where the slave node continuously replicates the master node. In case the master node fails, the slave mode is triggered and will stop replicating from the master node, execute a recovery and will become the master node.

ActiveMQ

The components of the Platform communicate with each other through a Message Queue. The Open Smart Grid Platform uses Apache Active Message Queue, which makes asynchronous communication possible between components. The components can register to the queues as consumers. In case a consumer (e.g. a server running a component of the platform) is down, the message will still be consumed by the remaining consumer(s). The Message Brokers can be used as a MasterSlave. In case the Master message broker is down, you get immediate fail-over to the slave without loss of messages.

Functional view

Image, functional layers overview

Starting architecture

The Functional view shows an overview of the most important functions of the system. The two images below show the starting architecture and functional reference architecture respectively.

Image, functional starting architecture

1. Web applications

2. Open Smart Grid Platform

3. Web services

4. Basic functions

5. Database

6. Communication infrastructure (CDMA/GPRS/Ethernet)

7. IP infrastructure

8. Open Street Light Protocol (OSLP)

9. Public Street Lighting Device (PSLD) or Sub Station Lighting Device (SSLD)

Functional Reference

This model partitions the system in seven functional clusters (vertically) which are shown on the system layers (horizontally). The circled numbers refer to image 1.

Vertical clusters:

• Device installation

• Device management

• Firmware management

• Configuration management

• Schedule management

• Ad-hoc control and status

• Monitoring

Horizontal System layers:

• Web applications

• HTTPS/SOAP communication

• Platform

• Open protocols

• Smart devices

Application Layering

The use of layers improves the separation of responsibilities. Each application contains the following layers:

• Presentation layer: responsible for providing information to users (persons and/or systems) and the handling of user requests

• Application layer: responsible for executing system tasks including authorization control

• Domain layer: responsible for the representation of the problem domain.

• Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc

Layers:

Authentication and authorization

The web server is configured with a SSL certificate to encrypt the incoming and outgoing communication. The SOAP Web service (Spring Framework web service) uses a Java Keystore and a certificate for each organization. Only organizations that are known within the platform are authorized to use the web service.

Application integration layer

For the several functional domains separate SOAP Web services are offered. This separation offers authorization per functional domain. Each of the web service components send a queue message to the corresponding domain component.

WSDL A separate WSDL is implemented for each functional cluster. All SOAP operations have a request object parameter and return a response object. For Synchronized Web Services the result is immediately included in the response. For asynchronous web services the response contains a correlation ID. This Correlation ID is to be used by the requester to receive the actual result from the platform. The following diagram is an example of such an asynchronous request.

Furthermore each SOAP message has a header which contains the user's organisation ID. This table displays an overview of the WSDL's including operations and fields in the request and response objects.

SOAP vs. REST

SOAP is chosen in the open smart grid platform web services over REST for the following reasons:

• REST is resources/data oriented (put, get, delete) while the open smart grid platform is function/method oriented

• SOAP has the advantage of having a contract (WSDL)

• SOAP has extensive security features that are being used in the open smart grid platform to meet the high security demands/requirements requested by e.g. the energy utilities

• Energy companies are generally not progressive in terms of technology. SOAP is acceptable for energy companies and REST is sometimes seen as new and insecure.

The benefits of REST (e.g. speed / less overhead) does not outweigh the benefits of SOAP. More general information on this topic can be found online.

Domain logic layer

For each functional domain business logic is implemented using a separate domain component. Common functionality like authorization should be abstracted to a shared component. Domain components receive queue messages from web service components and send queue messages to the open smart grid platform core component.

More information on the specific domains can be found in the domain chapter

Open Smart Grid Platform Core Services

The open smart grid platform core component receives queue messages from domain components. These messages from domain components are forwarded to a protocol adapter project. The open smart grid platform core component also offers logic for a protocol adapter project to send the response of a smart device back to a domain project. The Core component routes messages from domain adapter components to protocol adapter components and vice versa. The core layer also contains a workflow engine.

ERD's made with Valetina Studio

Overview of platform data model:

Data model explanation:

The platform will store as little data as possible. Generic (and domain specific) devices attributes are stored in core DB.

Protocol Layer

The open smart grid platform supports multiple protocols.

• OSLP (Open Street Light Protocol)

• DLMS/COSEM

• IEC61850

The protocols can use one of the security layers:

• TLS (Transport Layer Security encryption)

• SSL (Secure Sockets Layer encryption)

Other protocols can be easily added to the platform. If possible, we prefer protocols based on open standards. A comprehensive list of protocols that are currently supported can be found in the protocols chapter.

Protocol specific device attributes are stored in the protocol adapter DB

Queues

Open smart grid platform components connect to each other through message queues.

• Transactions on messages to and from the queues

• Messages are persisted on the queues

• Queues are clustered for reliability and speed

• By using queues, the open smart grid platform can be stateless

Smart devices

The open smart grid platform can connect to any device that supports one of the supported protocols. Smart devices can receive messages from or send messages to protocol adapter components. In case of SSLD's this is done using TCP/IP over mobile internet connections (e.g. GPRS, CDMA, etc.). The communication is encrypted using public key cryptography.

Introduction to Grid eXchange Fabric (GXF)

Grid eXchange Fabric was formerly known as the Open Smart Grid Platform (OSGP). GXF is an open, generic, scalable and independent 'Internet of Things' platform, which enables various connected smart objects in the public space to be easily controlled and monitored. Our platform allows the use of any (web)application and with any IP communication infrastructure.

Our goal is to stimulate the development of smart and sustainable solutions. Smart devices and smart apps play a central role in the development of smart grids and smart societies. The open smart grid platform software enables you to connect to thousands of devices, control them, and monitor their performance. This is done in an open and secure way, so you can use it for your own applications and devices, thereby reducing the time to market and decreasing development costs.

The names Open Smart Grid Platform and Grid eXchange Fabric will both be found in this document and in the code. In the future we will replace all references to the Open Smart Grid Platform and OSGP by Grid eXchange Fabric and GXF. During this transition period please read Grid eXchange Fabric if you see Open Smart Grid Platform.

Using the platform

The Grid eXchange Fabric is used in the following way:

• A user or operator uses one or more (web) applications to monitor and/or control devices

• The applications connect to the open smart grid platform via several web services which are divided into functional domains, i.e. Public Lighting, Smart Metering, Power Quality, etc. Third party developers can use the web services for the development or integration of new applications

• The platform handles all these application requests in a secure way and uses various functions and services to do so (e.g. authorization, authentication, device management, logging)

• For the 'translation' and communication of user/operator commands to the various smart devices, the platform uses multiple (open) protocols

• The platform supports various IP based data telecommunication technologies and protocols to communicate with the devices

Example use case for Grid eXchange Fabric

Use cases of Grid eXchange Fabric are only limited by your imagination. Here are some examples:

• Ad-hoc and scheduled Switching of Public Lighting

• Electrical Transportation

• Smart Metering

• Traffic Control (LED/matrix signs, traffic lights)

• Flexible load management ( solar energy and wind energy)

• Power Quality monitoring

Getting started

Basic Architecture

The basic architecture

Basic Overview

Layered architecture

The Open Smart Grid Platform environment consists of five layers:

1. Web services layer

2. Domain logic layer

3. Open Smart Grid Platform Core layer

4. Protocol layer

5. Device layer

Web services layer

In this layer the web services are exposed to the outside world. Applications can connect to the web services to implement the required functionality of the open smart grid platform. The web services are divided into functional domains, i.e. Public Lighting, Smart Metering, Power Quality, etc. Additional functional domains can be created.

Domain logic layer

Every functional domain has a separate set of web services and a corresponding domain logic block. In the domain logic block the business logic of that functional domain can be found. This is where a functional command will be translated into a generic intermediate format. For example, in the case of public lighting the command "Turn light on" will be translated into a command like "set switch(1) in closed position". In this layer it could also be decided that one functional command results in multiple commands to a device. The domain logic is closely related to the web services layers and can be added as well.

Open Smart Grid Platform core layer

In the core of the Open Smart Grid Platform the following generic functions are found:

• Device management

• Time synchronization

• Firmware management

• Workflow engine

• Device installation services

• Scheduler

• Device status monitoring

• Routing of device commands to appropriate device protocol

Protocol layer

The different protocol adapters are found in this layer. Here the generic intermediate format of a command for a specific device will be translated into the protocol message the device understands. This message will be sent to the device. A retry mechanism has been implemented to prevent communication failure in the case that the receiving end is temporarily unavailable. The listeners for messages initiated by a device are implemented here. Examples are the DLMS/COSEM protocol adapter for smart meters.

Devices

Any device in the public space with an Internet connection may be connected to the platform. The platform is independent of the device used, therefore this part of the set-up is not part of the platform.

Time behavior is mainly important in the Flexovl application when a lot of devices have to be addressed in a short period of time over a wireless network. Both latency and limited bandwidth have to be taken into consideration while demanding the coordinated on and off switching of the lighting, since we want to avoid the Christmas tree effect.

• Time synchronization: devices periodically register with the platform and receive a time.

• Protocol: because of the limited bandwidth an efficient protocol "protobuf" was selected.

Points of interest:

• Light metering messages

• When the SSLD's are disabled the PSLDs cannot be addressed

Because of these points of interest we use message queueing combined with a retry mechanism of delayed delivery.

The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.

The platform and devices use UTC time. The OSLP protocol between platform and devices uses UTC time as well.

Security

The following security measures can be used in a hosted environment:

Cloud security

• DDOS protection

• IPSEC VPN connections

• IP whitelisting

Most cloud environments support these features.

Operating System

• Hardened operating systems (according to Center of Internet Security)

Platform security

• Communication over TLS

• Firewalls between all servers and layers

• Certificates from a recognized Certificate Authority (CA)

• Audit trail on all actions throughout the platform

• Role based authorizations on specific functions of devices

• Access control

• Unique device identification

For every major release there will be a mandated security test initiated by Alliander.

In cooperation with the European Network of Cyber Security (ENCS) state of the art security measures were implemented.

• Security per device

• Security per application

• Security Certificates per Organisation and per device

• All communication is encrypted

Security measures: 1. Firewall in defined zone 2. Operating System Hardening 3. DDOS protection 4. Replay attack prevention 5. Private encryption key per device 6. Certificates from a Certificate Authority 7. Encryption via Elliptic Curve DSA 8. IPSEC VPN for CDMA and GPRS 9. Unique device identification 10. Unique CDMA modem number 11. Role based authorizations on functions and devices

Encryption

An analysis of safety aspects has led to the decision that the safety of the whole system will be realized by proven technology based on asymmetrical coding (also known as public-key encryption).

Authentication of web applications

Two-way SSL will be used between web applications and the Open Smart Grid Platform to verify the identities for both client and server. User organisations are responsible for the administration of the identity of and access to their web applications. The web applications feature a login page. After successful login the user is linked to an organisation. Passwords will be stored with encryption. The organisation ID will be sent in each message to the Open Smart Grid Platform and will be verified by the SSL certificate.

Algorithms

Only public encryption Algorithms will be used. Due to performance limitations (of the devices) and recommendations from The European Network for Cyber Security (ENCS) Elliptic Curve DSA with 256-bit-keys was selected. This improves the security and efficiency over the 1024 bit RSA algorithm. Messages can be smaller and less processor capacity is needed. The key length of Elliptic Curve DSA is similar to the 3072 bit key length of RSA.

Note: Even though the open smart grid platform uses ECDSA to secure the OSLP, other encryptions may be used as well. The RSA Algorithm is still supported if preferred. This is a flexible configuration option.

Private APN

A private APN is used for linking to mobile data communication infrastructures.

Logging

• Every action to and from devices is logged in the audit trail

• Messages from unknown devices will be denied (and logged)

This chapter describes the results of a performance test, to give potential users an indication of the system requirements for the platform.

The Platform was tested with the following AWS setup:

Systems:

• Specifications Component Server: 2 CPU's, 8 GB RAM

• Specifications Database Server: 1 CPU, 2 GB RAM

Setup:

• Front-end: 2x Component Server

• Middle-end: 2x Component Server

• Back-end: 2x Component Server

• ActiveMQ Front-Middle: 1x Component Server

• ActiveMQ Middle-Back: 1x Component Server

• Databases: 1x DB Server

For the test to succeed, two requirements were to be met:

1. Switch 10.000 simulated OSLP devices under 5 minutes.

2. Switch 40.000 simulated OSLP devices under 5 minutes.

The results showed that both tests succeeded.

The Core layer of the Open Source Grid Platform is responsible for Validation, Translation, Authorisation and Routing of request messages. It also contains all the Domain Objects.

The core layer consists of two components:

• osgp-domain-core: Shared Domain objects, services, repositories, etc. These classes are used through the entire platform.

• osgp-core: Logic for routing domain requests, scheduling, retrying, etc.

General Package structure: osgp-domain-core

• entities: Defines the entities used for persistence.

• exceptions: Domain specific exceptions reside here.

• repositories: Repositories that contain logic for persisting entities.

• services: Domain services that reference a repository.

• specifications: Interfaces that define specifications for Devices and Events.

• validations: Validators and constraints.

• valueobjects: Definitions of the Domain Objects.

General Package structure: osgp-core

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- OsgpCoreInitializer

  -- DomainMessagingConfig

  -- PersistenceConfig

  -- ProtocolMessagingConfig

  -- SchedulingConfig

• services: Services that process device requests/ responses. Checks for authorization, and if the request is supported by the platform, it will be routed to the appropriate protocol adapter.

• tasks: Contains task scheduler logic.

domain.model

These packages contain interfaces for the Services.

• domain: Interfaces for the Domain services.

• protocol: Interfaces for the Protocol services.

infra.jms

• domain: Contains Messages, MessageListeners and MessageProcessors for Domain related messaging.

• protocol: Contains Messages, MessageListeners and MessageProcessors for Protocol related messaging.

This chapter gives a more technical overview. It describes each layer of the platform by giving an overview of its packages and the code it contains. Furthermore it describes how a message proceeds through the platform. If you are planning on adding your own Domain Adapter or Protocol Adapter, it will be useful to read this chapter to get a feeling of how the Platform has been built.

A Request through the Platform

The picture below depicts an example of a request (OSLP SetLight request) proceeding through the platform to a device.

• A web request enters the platform at its EndPoint, which in turns calls the RequestService. The RequestService checks if the organisation in the request is authorized, creates the request message and sends it to the MessageSender which in turn puts it on the queue of the Domain Adapter.

• In the Domain Adapter, the incoming message is processed in the MessageProcessor, which in turn calls the Request Service. Here the message is converted to a DTO object. The CoreRequestMessageSender puts the message on the Core Queue.

• The MessageListener in Core receives the message. The DeviceRequestMessageService contains generic functionality such as Authorization, Validation, etc. Once these procedures are completed, the message is routed to the appropriate protocol adapter.

• In the Protocol Adapter the message is received by the MessageListener. It is processed through the MessageProcessor and OslpDeviceService. The request eventually ends up in the OslpChannelHandler, where the actual Protocol Request to the device is made.

For a detailed description of each layer, please take a look at a more detailed description of each layer in this chapter.

Configuration files

The Platform uses property files for certain settings (such as JMS settings, Persistence settings, etc.). These files are stored in property files which can be found in the Config repository on Github. These files are sym linked to /etc/osp/, where the Platform (through reference in context.xml) looks for the property files.

The Domain Adapters are responsible for receiving requests from the Web Services layer, and delivering them to the Core layer. The Domain Layer mainly contains MessageProcessors and Services for request handling.

The Core/Admin components contains the shared functionality, while the Domain components contain additional domain specific functionality.

At the moment the Platform uses the following Domain Adapters:

Generic

• Core - osgp-adapter-domain-core: Contains Core (common) functionality; AdHocManagement, FirmwareManagement, etc.

• Admin - osgp-adapter-domain-admin: Contains Admin functionality, e.g. DeviceManagement.

Domain

• Public Lighting - osgp-adapter-domain-publiclighting: Contains functionality for the Public Lighting Domain.

• Smart Metering - osgp-adapter-domain-smartmetering: Contains functionality for the Smart Metering Domain.

• Tariff Switching - osgp-adapter-domain-tariffswitching: Contains functionality for the Tariff Switching Domain.

• Microgrids - osgp-adapter-domain-microgrids: Contains functionality for the Micro Grids domain.

• Distribution Automation - osgp-adapter-domain-distributionautomation: Contains functionality for the Distribution Automation domain.

General Package structure

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- DomainAdapterInitializer

  -- MessagingConfig

  -- PersistenceConfig

• mapping: Custom Orika converters for mapping to/from DomainObjects/DTO Objects.

• services: Contains most of the domain logic, related to the specific services of the adapter. The service classes converts DTO objects to Domain objects (or vice versa), and put the request on the Core queue through the JMS classes.

infra

• jms.core: Inbound/outbound messages from/to the Core layer.

  This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Core Queue, or receiving and processing responses from Core.

• jms.ws: Inbound/outbound messages from/to the web services layer.

  This package contains Messages, MessageListeners, MessageSenders and MessageProcessors for sending requests to the Web Services Queue, or receiving and processing responses from the web services layer.

The Protocol Adapters are responsible for the actual communication to and from a device. They usually operate in a certain domain, and might use common/ generic functions from the platform.

The Open Smart Grid Platform currently has the following protocol adapters:

• Protocol-Adapter-DLMS: Smart Metering

• Protocol-Adapter-IEC61850: Public Lighting, Micro Grids and Distribution Automation

• Protocol-Adapter-OSLP: Public Lighting and Micro Grids

General package structure

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- MessagingConfig

  -- OsgpProtocolAdapterOslpInitializer

  -- OslpConfig

  -- OslpPersistenceConfig

• mapping: Custom Orika converters.

• services: Contains the (functional) services that control communication from (and to) the device. Also persists requests and responses, and deals with security. Actual communication is done through the classes in the infra package.

device

Contains the Protocol Adapter Objects used for the Protocol Adapter.

• requests: Objects representing the requests that are supported by this adapter.

• responses: Objects representing the responses that are supported by this adapter.

domain

• entities: Entities used for persistence.

• repositories: Repositories used for persistence.

exceptions

Contains the exceptions that might be thrown while communication with a device, e.g. ValidationException, MessageRejectedException, etc.

infra

This package contains all the code for communication through JMS (Platform) and Networking (Device).

• messaging: Contains the JMS Messages, MessageListeners, MessageSenders and MessageProcessors.

• networking: Contains the classes that are responsible for connecting to a device using a certain protocol.

services

Services used to check the request status.

The open smart grid platform use-cases are strongly related to the open smart grid platform domains. Up-to-date information on use-cases can be found on the Grid eXchange Fabric website.

Platform

• Apache ActiveMQ: Open source messaging server, used to relay messages between components of the open smart grid platform. ActiveMQ is an open source message broker written in Java and a full Java Message Service (JMS) client. It provides "Enterprise Features" which in this case means fostering the communication from more than one client or server.

• Apache HTTP server: Web server, used as front for Apache Tomcat.

• Apache Tomcat: Provides a "pure Java" servlet container for Java code to run in.

• pgAdmin-III: PostgreSQL administration and management tools.

• Protobuf (Google Protocol Buffers): A language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more.

• Flyway: Agile database migration framework for Java

• HikariCP: JDBC connection pool

• Hibernate: Object/Relational mapping

• Netty: Network application framework for protocol servers & clients

• OpenMUC: Library implementing the IEC61850 and DLMS/COSEM communication standard

• Orika: Java bean mapping

• Spring: Application development framework. Several Spring libraries are used, including Spring Data, Spring Security and Spring WS.

• Puppet: Application for Automatically delivering, operating and securing your infrastructure

Development

• Bower: Package manager for Javascript packages. Web applications consist of various components; frameworks, libraries, assets, utilities, and rainbows. Bower manages all these things for you.

• Eclipse: IDE for developing software.

• FileZilla: FTP application.

• Git: Version control system.

• NodeJS: Tooling suite with various Javascript tools.

• NPM: Package manager for the NodeJS Javascript applications.

• Putty: A free and open-source terminal emulator, serial console and network file transfer application.

• Vim: Source code editor.

• Apache Maven: Software project management tool.

• GIT & GitHub: Source code management.

Testing &QA

• Apache JMeter: Application designed to load test functional behaviour and measure performance.

• Cucumber: automated acceptance testing framework.

• Gherkin: DSL for acceptence testing framework.

• Sonarqube: Quality management platform.

• SoapUI: Functional testing tool for testing web services.

• JUnit: Unit testing.

• Mockito: A Mock framework for Unit testing.

The following table presents an overview of the components and the most important technical choices per component.

Throttling is used to limit the flow into communication channels that need it. The GXF platform has throttling support for CDMA and SMS/texting.

CDMA throttling

The CDMA network uses base transceiver stations (BTS) with cells for each BTS.

Throttling occurrs on the number of concurrent connections per BTS/cell. GXF offers a throttling service that hands out permits per BTS/cell while maintaining the configurable limit.

An application will typically use this service like this:

SMS/texting throttling

SMS can be used to wake up certain types of devices. Throttling in this context means limiting the number of messages per second to a configurable rate.

To get started quickly, a Vagrant Installation Guide has been created and a guide for Manual Installation.

The goal of the installation manual is to control a simulated OSLP device through the Platform. Below, is a summary of all steps involved. See the next chapters for a detailed guide with screenshots. Please follow the steps carefully.

A summary of the steps involved:

• Creating a virtual machine using Vagrant and Virtual Box

• Run the puppet script (part of the Vagrant installation), or complete the steps manually (Manual installation)

• Importing Maven Projects into Eclipse

• Creating an Apache Tomcat9 Server

• Setting Up Apache Tomcat9 Server Context

• Deploying all open smart grid platform components to an Apache Tomcat9 Server

• Starting Apache ActiveMQ

• Starting Apache Tomcat9 Server

• Creating the 'test-org' organization in the Database

• Setting up SoapUI

• First SOAP Requests to add a device to the open smart grid platform

• Opening Device Simulator to add a device

• Registering a device

• Using 'SetLight' SOAP Request to switch the light on

To install the platform you can use one of the following procedures.

1. The Vagrant Installation. This procedure creates and installs a complete image with the Open Smart Grid Platform pre-installed, including all the tools such as Maven, Eclipse, SoapUI, etc.

2. The Manual Installation. Follow this guide if you want to install the Open Smart Grid Platform yourself.

This chapter gives an overview of the principles used defining and implementing the architecture. The following principles were applied:

• Layering

• Domain driven design

• Dependency inversion principle

• Behavior driven development

Layering

The use of layers improves the separation of responsibilities. Each application contains the following layers:

• Presentation layer: responsible for providing information to users (persons and/or systems) and the handling of user requests

• Application layer: responsible for executing system tasks including authorisation control

• Domain layer: responsible for the representation of the problem domain.

• Infrastructure layer: responsible for technical matters supporting other layers. For instance persistence, messaging, etc

Image, Layers:

1. Audit logger

2. Web Services

3. Functions

4. Queue

5. Workflow engine

6. Protocol framework

7. Protocol implementations

8. Workflow engine

9. Queue

10. Communication

Domain driven design (DDD)

Domain-driven design focuses on the problem domain. DDD's starting point is creating an optimal model for a specific problem domain by having a common language and constructive collaboration between technical and domain experts.

DDD uses the following building blocks:

• Entity: An object not identified by its attributes but by its own identity.

• Value Object: an object with attributes but has no own identity.

• A collection of objects surrounding a specific root entity (or aggregate root). To ensure consistency objects in the aggregate can only be addressed through the aggregate root.

• Service: Contains instructions not related to a specific object.

• Repository: Serves as a collection for fetching and saving objects. Creates an abstraction for actual persistent implementations.

• Factory: Contains methods to create domain objects.

Dependency inversion principle

The dependency inversion principle promotes an independent connection by inverting dependency relations. This ensures that the domain model can be very 'clean' without knowledge of the underlying infrastructure (POJO classes). The Spring framework is used to implement the Dependency Inversion principle.

Behavior driven development (BDD)

Behavior driven development is a way of programming that first describes behavior in user stories and then implements this in code. The user stories contain scenarios with acceptation criteria that can be automated. This creates a complete test suite for the whole system.

For the application of BDD the following frameworks are used:

• Cucumber and Gherkin, automated acceptance testing, based on scenarios from stories.

Testing the platform using the Web Demo

This chapter will describe the steps needed to test the Open Smart Grid Platform using the Web Demo Application.

If you followed the steps from the 'Platform Setup' then you can skip the following installation steps, because the Web Demo will already be available.

Installing the Web Demo

• Open Eclipse and import the following folder as Maven project /home/dev/Sources/OSGP/open-smart-grid-platform/public-lighting-demo-app/

• Add the PublicLightingDemoApp components to the Tomcat Server.

• (Re)Start the Tomcat Server.

Creating a device

To access the Demo App go to the following URL: https://localhost/web-demo-app/

If you encounter an Untrusted Connection page, go to 'I Understand the Risks' -> Add Exception.. -> Confirm Security Exception

Click the Add a Device button in the Menu bar, and enter SSLD_000-00-01 at the Device Identification field and press Submit.

The following screen will appear, it shows that the device has been successfully added to the Platform.

Opening Device Simulator to Add a Device

In order to be able to use the SSLD-000-00-01 Device, the device needs to be simulated in the Device Simulator. To do this we have to create it. In the Firefox Browser, open the Device Simulator by going to the following URL:

https://localhost/web-device-simulator/devices

Fill out the fields like this:

• Device Identification: SSLD_000-00-01

• IP Address: 127.0.0.1

• Device Type: SSLD

• Protocol: OSLP_ELSTER

You should return to the Devices screen and see the message "Device with identification SSLD_000-00-01 was created."

Registering a Device

Then click the 'Confirm Registration' button. The message should read: "Device with identification SSLD_000-00-01 was confirmed to be registered."

Using 'SetLight' Switch the Light On

Now that the Device is known in the platform, and simulated in the Device-Simulator, the device can be used. Let's switch on the Light. Go to the Demo App and press the List button in the Menu. A list should appear, showing the device that has just been added using the Add Device button.

Click on the Manage button to access the Device Details.

Switch on the Light by setting the Light Value to 100 and by checking the 'LightOn' checkbox (as shown in the screenshot below)

Hit submit to submit the request to the Platform. The following screen should appear:

In the home screen of the OSLP device simulator, the lightbulb should light up for SSLD_000-00-01. This means that the request succeeded.

This step also concludes the installation manual.

Manual Installation

This chapter describes the steps for a manual installation (eg. not using the vagrant script and puppet scripts). This chapter is for developers who would like to have more control over the installation procedure.

With the increased control come increased risks of things not working with the versions or configuration of the software involved with the OSGP environment. If you run into issues, you may find clues in the puppet scripts about versions and modifications to the configuration of installed software.

Note

• Skip this chapter if you followed the Vagrant installation! You can continue with next chapter: Setup the Open Smart grid Platform

Operating System

The Open Smart Grid Platform runs on a Linux environment. It is recommended to set up a machine running Ubuntu.

Software and tools

The Open Smart Grid platform needs the following software and tools installed/downloaded:

• Java 8 openjdk-8

• PostgreSQL and pgAdmin 3

• Git

• Maven

• ActiveMQ

• Tomcat

• Apache HTTPD

• SoapUi

• Eclipse IDE for Java EE Developers

• Google Protocol Buffers: protobuf-compiler, libprotoc7 and libprotobuf7

• PostgreSQL JDBC driver

Settings

User

It is recommended to create a 'dev' user, because some scripts contain hard coded references to this 'dev' user. It is possible to skip this step, but then some of the scripts will have to be adjusted manually.

Tomcat

• Place the PostgreSQL JDBC driver jar in the Tomcat lib directory.

• Change permissions of Tomcat Config files to 644 in the Tomcat conf directory.

Apache HTTPD

• Enable mod_ssl by running the following command:

  Copy

  a2enmod ssl

• Enable proxy_ajp by running the following command:

  Copy

  a2enmod proxy_ajp

Java

• Make sure the JAVA_HOME var is set, and points to openjdk-8.

Cloning Sources

Clone the following repo's, it is recommended to create a Sources/OSGP directory in /home/dev/ since some scripts contain hard coded references to those folders.

git clone https://github.com/OSGP/Config.git /home/dev/Sources/OSGP/Config
git clone https://github.com/OSGP/open-smart-grid-platform.git /home/dev/Sources/OSGP/open-smart-grid-platform
git clone https://github.com/OSGP/Documentation.git /home/dev/Sources/OSGP/Documentation

Make sure you are on the development branch (default).

Creating directories and symlinks

Create the following directories:

• /var/log/osp/logs

• /etc/osp/

Make the dev user (or equivalent) the owner of the log directory with rwx permission. Give the other users read and execute permission.

Execute the script /home/dev/Sources/OSGP/Config/scripts/create-symlinks.sh

Note This script uses hard coded references to /home/dev/Sources/OSGP/*, if you used a different user, please edit the script before executing it.

The script will make symlinks to certificates, to Apache HTTP server configuration and copy configuration settings as samples to locations where these properties may be overridden.

Initiating the Database

To create the database run the following command (Change /home/dev/ in case of no dev user)

sudo -u postgres /usr/bin/psql -p 5432 -f /home/dev/Sources/OSGP/Config/sql/create-users-and-databases.sql

And create a backup of the pg_hba.conf file (modify if your version of PostgreSQL is different)

cp -p /etc/postgresql/9.4/main/pg_hba.conf /etc/postgresql/9.4/main/pg_hba.backup

Finally, reload the postgresql service:

service postgresql reload

Set up the Open Smart Grid Platform

Continue with setting up the Open Smart Grid Platform by following the Set up the Open Smart Grid Platform Guide

Platform uses single device calls.

In order to add a device to the platform, call the add device method in the device installation web service (for a specific domain) for each device.

Implementation may differ for different device types

Each device type may require its own set of attributes, security settings, etc. Each device type may have its own registration mechanism.

• To add a DLMS Device to the Platform, take a look at the documentation of the Soap call AddDevice as defined in the SmartMeteringInstallation.wsdl.

• To add an OSLP Device to the Platform, the Soap call defined in CoreDeviceInstallation can be used, or the UpdateKey Request.

Please take a look at the chapter Testing the open smart grid platform in the installation manual for a detailed guide of how to add a OSLP device to the platform.

Additional Device actions

In the Domain Chapter of the documentation more information of the Web Service calls can be found for Adding devices, setting configuration, authorizations, schedules, firmware updates, etc.

Setting Up the Open Smart Grid Platform Development environment

This chapter describes all the steps needed to finalize the open smart grid platform development environment.

Lombok

The platform uses Lombok annotations to generate extra Java methods. Without Lombok the project is not imported correctly by Maven and will not run. If you used the Vagrant installation method it should already be installed. To check if Lombok is properly installed to go Help > About Eclipse IDE and scroll down. Here you will see: Lombok <version> "<version name>" is installed. https://projectlombok.org/. If not follow this guide to install Lombok.

Importing Maven Projects into Eclipse

Open Eclipse by clicking the shortcut on the Desktop and import the projects.

Go to File -> Import -> Existing Maven Projects, browse to folder /home/dev/Sources/OSGP

Import the projects from location /home/dev/Sources/OSGP/open-smart-grid-platform.

Creating an Apache Tomcat Server

In Eclipse go to Window -> Open Perspective -> Debug

In the 'Debug' perspective, go to the 'Servers' view and add a new Apache Tomcat server, Tomcat is available in the folder /home/dev/Tools/tomcat (or in another location if you didn't set up a VM using Vagrant, the latest version usually works fine).

Click on Next

Click on Finish

Make sure that Tomcat is using the correct Java Runtime Environment:

Click on 'Runtime Environment', 'Installed JREs' and click 'Add'. Choose 'next' with 'Standard VM' highlighted. Click 'Directory' and in the folder browser that is shown, use the key combination 'Ctrl + h' to show hidden files. Choose '.sdkman/candidates/java/current' and select 'Open' and 'Finish'. In the 'Installed JREs' screen, unselect the default JVM (which will be an Eclipse hotspot JVM) and select '/home/dev/.sdkman/candidates/java/current' then close the screen by clicking 'Apply and Close'. The last step is to choose the new JRE which was just added in the dropdown of the 'Runtime Environment' screen by selecting '17.0.5'. Click 'Finish' to apply.

Setting Up Apache Tomcat Server Context

All modules contain their own context.xml. In the module specific context.xml are the environment variables defined where the global and module specific configuration files are located. Default they will point to a location in /etc/osp/.

Deploying all Open Smart Grid Platform components to Apache Tomcat Server

Starting Apache ActiveMQ

Continue with starting Apache ActiveMQ. If you installed an environment as described with Vagrant, you can double click the ActiveMQ shortcut on the desktop.

Alternatively you can open a terminal and run the executable manually by using the following command: (the executable can be found in the folder /home/dev/Tools/activemq/bin)

./activemq console

Starting Apache Tomcat Server

With ActiveMQ running, the Tomcat server can be started. Go to Eclipse, go to the Servers tab in the Debug view, and right click on the Tomcat server and select 'Start'.

note: In case of an error starting up for the very first time, try and start up only the module: 'osgp.core' first. This makes sure the database scripts are executed.

Probe

This is an optional program that shows the status of the Tomcat resources in real time. To install Probe you can follow this guide. Note that you need to add the Tomcat users in the guide's Security part in the tomcat-users.xml in your Eclipse environment. To download the war file go to: https://github.com/psi-probe/psi-probe/releases. You need to copy the war file to: /home/dev/<your Eclipse workspace>/.metadata/.plugins/org.eclipse.wst.server.core/tmp0/webapps.

Starting pgAdmin III and Connect to PostgreSQL

Open pgAdminIII and configure a connection: choose the 'Add a connection to a server.' and fill out the fields using

• Host: localhost

• Port: 5432

• Username: osp_admin

• Password: 1234

Creating the 'test-org' Organization (in database osgp-core)

Run the script in /home/dev/Sources/OSGP/Config/sql/create-test-org.sql to insert 'test-org' organization into the organisation table of the osgp_core database.

psql -U osp_admin -h localhost -d osgp_core -f /home/dev/Sources/OSGP/Config/sql/create-test-org.sql

If asked for a password, enter 1234

Go back to PgAdmin III, expand servers, select localhost -> databases -> osgp_core -> Schemas -> public -> Tables. Right click the organisation table and select to view data for the top 100 rows. Confirm that the test-org organisation has been added to the Database.

Now that everything has been set up, continue to the next chapter to start testing the Platform by sending it some requests.

The Open Smart Grid Platform is designed and built for scalability and reliability

• Messages will never get lost, In the worst case scenario, a message will be sent to the dead letter queue.

• Any layer of the platform can be independently scaled up- and down

• Adding servers can be done runtime

• It can run in an active-active setup over multiple servers and data centers. In our cloud hosted setup even over sets of data centers in different countries.

There are two procedures for testing the Open Smart Grid Platform.

1. SoapUI. Create and send Soap requests to the Platform to manage a (simulated) light.

2. PublicLighting Demo App. Use the Demo App to send requests to the Platform to manage a (simulated) device.

The Web Services layer contains the web services that are used to communicate with the Platform. The Open Source Smart Grid Platform uses the Simple Object Access Protocol or SOAP to expose its interfaces. The Web Services Adapter receives requests and sends those to the Domain Adapter. An incoming request is converted to a Domain Object, and put on the MessageQueue of the Domain layer. The Web Services layer also has a queue for incoming responses from the Domain adapter.

Each domain of the Platform has its own web services:

Generic

• Core - osgp-adapter-ws-core: Contains the Core (common) web services.

• Admin - osgp-adapter-ws-admin: Contains the Admin web services.

• Shared - osgp-adapter-ws-shared: Contains shared endpoints, such as header authorization

• Database - osgp-adapter-ws-db: Contains repositories for persistence.

Domain

• Public Lighting - osgp-adapter-ws-publiclighting: Contains the Public Lighting web services.

• Smart Metering - osgp-adapter-ws-smartmetering: Contains the Smart Metering web services.

• Tariff Swithcing - osgp-adapter-ws-tariffswitching: Contains the Tariff Switching web services.

• Microgrids - osgp-adapter-ws-microgrids: Contains the Micro Grids web services.

• Distribution Automation - osgp-adapter-ws-distributionautomation: Contains the Distribution Automation web services.

For a description of the WSDL's see the Domain Chapter.

General Package structure

A description of the general package structure of a web service component.

application

• config: Contains the configuration files for the Component. Uses the property files in /etc/osp/.

  -- ApplicationContext

  -- AdapterInitializer

  -- MessagingConfig

  -- PersistenceConfig

  -- WebServiceConfig

• exceptionhandling: Exceptions are defined here.

• mapping: Custom Orika converters.

• services: Contains services used by the domain, such as AdHocManagement. These are called by the end points and convert the request to a Domain Object and put the request on the domain message queue using the JMS classes.

endpoints

• EndPoints for the web services: contain a reference to a service that proceeds with handling the request.

infra

• jms: contains the JMS classes such as:

  -- MessageSender(s)

  -- MessageType

  -- ResponseMessageFinder

webapp

The WSDL and schema definitions can be found under main/webapp/WEB_INF/wsdl.

Basic configuration

In order to use git correctly you need to execute the following commands in a terminal if you haven't done so already:

git config --global user.name "your full name"
git config --global user.email yourEmailAddress

Contributors License Agreement (CLA)

GXF is covered by LFE's CLA. You have to sign/agree to it in order to commit changes. Most likely you will have to sign the individual agreement, unless you work for a licensed company.

Your pull request will detect if you haven't signed it yet and instructions are given on how to continue.

Developer Certificate of Origin (DCO)

GXF uses GitHub's DCO application, so you need to sign-off your commits.

Your pull request will detect if you haven't signed-off your commits and instructions are given on how to continue.

Command line

Just add -s to git commit ....

Note this is the lower case -s (short for --signoff).

With git merge ... you can add --signoff as the shorter -s is used to set the merge strategy.

Eclipse

IntelliJ

We have prepared a full set of configuration (properties files, certificates, key store, ...) which is available in our Config repository. This configuration can be used to setup a trial environment.

Testing the platform

This chapter will describe the steps needed to test the Open Smart Grid Platform.

Setting Up SoapUI

Start SoapUI by double clicking the shortcut on the Desktop or run it manually by typing the following command in a terminal:

/home/dev/Tools/SoapUI/bin/soapui.sh

Go to File -> Preferences -> SSL Settings, and browse for the KeyStore to /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/test-org.pfx and fill out the password (the password is 1234)

Go to WSDL Settings and check 'Generate Example Values in New Requests' and 'Generate Comments with Type Information in New Requests'

Add the SoapUI projects to SoapUI

There are several SoapUI project prepared, see /home/dev/Sources/OSGP/Config/soapui/. Import all SoapUI projects present in the folder mentioned above. Below, 2 projects are shown as examples.

Adding the 'Admin' Soap project

Import the 'admin' project by clicking File -> Import project. Browse to /home/dev/Sources/OSGP/Config/soapui/, select 'admin-soapui-project.xml' and click open.

Alternatively you can create the 'admin' project yourself by following the steps below:

• Create a new SOAP Project and call it 'admin'

• Open the Project View by double-clicking on the 'admin' project. Go to 'WS-Security Configurations' and select the 'Keystores' Tab. Click on the '+' to add the test-org.pfx in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

• Fill out the password (1234) and click Ok and close the Project View window.

• Right click the 'admin' project and choose 'Add WSDL'. Enter the following URL in the WSDL Location field:

  Copy

  /home/dev/Sources/OSGP/open-smart-grid-platform/osgp/shared/osgp-ws-admin/src/main/resources/AdminDeviceManagement.wsdl

• Make sure the box 'Create sample requests for all operations' is checked, and click OK.

Adding the 'Public Lighting' Soap project.

Import the 'public-lighting' project by clicking File -> Import project. Browse to /home/dev/Sources/OSGP/Config/soapui/, select 'public-lighting-soapui-project.xml' and click open.

Alternatively you can create the 'public-lighting' project yourself by following the steps below:

• Open the Project View by double-clicking on the 'public-lighting' project. Go to 'WS-Security Configurations' and select the 'Keystores' Tab. Click on the '+' to add the test-org.pfx in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

• Right click the 'public-lighting' project and choose 'Add WSDL'. Enter the following URL in the WSDL Location field:

  Copy

  /home/dev/Sources/OSGP/open-smart-grid-platform/osgp/shared/osgp-ws-publiclighting/src/main/resources/PublicLightingAdHocManagement.wsdl

First SOAP requests to add a device to the open smart grid platform

Before sending the request, the test-org.pfx should be added as SSL Keystore: Go to the properties interface for the request (bottom left of the screen, after selecting 'Request 1' under UpdateKey in the 'admin' project'), and choose test-org.pfx from the drop-down box.

Note

• This has to be done for each request!

An SSLD needs to be added to the platform, as well as a manufacturer and a public key for the SSLD. A couple of steps need to be performed to realize this.

1 Add manufacturer 2 Add device model 3 Add SSLD 4 Setup a protocol for the SSLD to use 5 Set the public key for the SSLD (in case of OSLP)

The AddManufacturer function adds a new manufacturer to OSGP. All devices are coupled to a manufacturer.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/common/firmwaremanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:AddManufacturerRequest>
         <ns1:Manufacturer>
            <!--type: int-->
            <ns1:Id>3</ns1:Id>
            <!--anonymous type-->
            <ns1:Code>MAN</ns1:Code>
            <!--anonymous type-->
            <ns1:Name>Manufacturer01</ns1:Name>
            <!--type: boolean-->
            <ns1:UsePrefix>false</ns1:UsePrefix>
         </ns1:Manufacturer>
      </ns1:AddManufacturerRequest>
   </soapenv:Body>
</soapenv:Envelope>

The AddDeviceModel function adds a new device model to OSGP. All devices are coupled to a device model.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/common/firmwaremanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:AddDeviceModelRequest>
         <ns1:DeviceModel>
            <ns1:Manufacturer>MAN</ns1:Manufacturer>
            <!--Optional:-->
            <ns1:ModelCode>MOD01</ns1:ModelCode>
            <!--Optional:-->
            <ns1:Description>Device model MOD01.</ns1:Description>
         </ns1:DeviceModel>
      </ns1:AddDeviceModelRequest>
   </soapenv:Body>
</soapenv:Envelope>

The AddDevice function adds a new SSLD to OSGP. The device is coupled to a device model and a manufacturer.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/deviceinstallation/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:AddDeviceRequest>
         <ns1:Device>
            <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
            <!--Optional:-->
            <ns1:Owner>test-org</ns1:Owner>
            <!--Optional:-->
            <ns1:Activated>false</ns1:Activated>
            <!--Optional:-->
            <ns1:HasSchedule>false</ns1:HasSchedule>
            <!--Optional:-->
            <ns1:PublicKeyPresent>false</ns1:PublicKeyPresent>
            <ns1:DeviceModel>
               <!--anonymous type-->
               <ns1:Manufacturer>MAN</ns1:Manufacturer>
               <!--Optional:-->
               <!--anonymous type-->
               <ns1:ModelCode>MOD01</ns1:ModelCode>
               <!--Optional:-->
               <!--anonymous type-->
               <ns1:Description>Test device.</ns1:Description>
            </ns1:DeviceModel>
         </ns1:Device>
      </ns1:AddDeviceRequest>
   </soapenv:Body>
</soapenv:Envelope>

The function UpdateDeviceProtocol sets a protocol for a device.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:UpdateDeviceProtocolRequest>
         <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
         <ns1:ProtocolInfo>
            <ns1:Id>4</ns1:Id>
            <ns1:Protocol>OSLP ELSTER</ns1:Protocol>
            <ns1:ProtocolVersion>1.0</ns1:ProtocolVersion>
         </ns1:ProtocolInfo>
      </ns1:UpdateDeviceProtocolRequest>
   </soapenv:Body>
</soapenv:Envelope>

The UpdateKey function of the admin webservice sets a public key for a device. Double click 'Request 1' under UpdateKey in the 'admin' project. Add the following request:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:UpdateKeyRequest>
         <!--type: Identification-->
         <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
         <!--type: string-->
         <ns1:PublicKey>MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEFhUImXFJdqmputquVAc2CPdnn9Ju00M3m/Ice7wABNN+oAYKQbw/OceqvZmFF1+r4nO/vCm/f1JO5nEorE2jNQ==</ns1:PublicKey>
         <!--type: long-->
         <ns1:ProtocolInfoId>1</ns1:ProtocolInfoId>
      </ns1:UpdateKeyRequest>
   </soapenv:Body>
</soapenv:Envelope>

After the SSLD has been added, let's see if the function FindAllDevices shows the SSLD. Continue with the FindAllDevices request from the public-lighting project. Since this is not the same project, we have to change the endpoint; in this case in https://localhost:443/osgp-adapter-ws-publiclighting/publiclighting/adHocManagementService/. Do not forget to set the SSL keystore in the Request Properties. Use the following parameters in the request:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:FindAllDevicesRequest>
         <!--type: int-->
         <ns1:Page>0</ns1:Page>
      </ns1:FindAllDevicesRequest>
   </soapenv:Body>
</soapenv:Envelope>

Opening Device Simulator to Add a Device

In order to be able to use the SSLD-000-00-01 Device, the device needs to be simulated in the Device Simulator. To do this we have to create it. In the Firefox Browser, open the Device Simulator by going to the following URL:

https://localhost/web-device-simulator/devices

Fill out the fields like this:

• Device Identification: SSLD_000-00-01

• IP Address: 127.0.0.1

• Device Type: SSLD

• Protocol: OSLP ELSTER

Registering a Device

Using 'SetLight' SOAP Request to Switch the Light On

Now that the Device is known in the platform, and simulated in the Device-Simulator, the device can be used. Let's switch on the Light. Using SoapUI, click on Request 1 under SetLight at the public-lighting project. Set the following parameters in the request (And do not forget to set the Keystore in the request properties):

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:SetLightRequest>
         <!--type: Identification-->
         <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
         <!--1 to 6 repetitions:-->
         <ns1:LightValue>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:Index>0</ns1:Index>
            <!--type: boolean-->
            <ns1:On>true</ns1:On>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:DimValue>100</ns1:DimValue>
         </ns1:LightValue>
      </ns1:SetLightRequest>
   </soapenv:Body>
</soapenv:Envelope>

The last request concerns the response from the previous SetLight request. In SoapUi open Request 1 under 'GetSetLightResponse' in the 'public-lighting' project. Set the following parameters in the request (And the keystore in the request properties). Make sure to replace the CorrelationUid with the value from the respons from the SetLight request.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:SetLightAsyncRequest>
         <ns1:AsyncRequest>
            <!--type: CorrelationUid-->
            <ns:CorrelationUid>test-org|||SSLD_000-00-01|||20160721083641940</ns:CorrelationUid>
            <!--type: Identification-->
            <ns:DeviceId>SSLD_000-00-01</ns:DeviceId>
         </ns1:AsyncRequest>
      </ns1:SetLightAsyncRequest>
   </soapenv:Body>
</soapenv:Envelope>

Note

• Do not forget to set the CorrelationUid to value in the response you received from the setLight request.

The server replied Ok, indicicating that the SetLight request has been processed succesfully.

This step also concludes the installation manual.

Documentation Publication

This documentation is available in multiple formats.

Web

PDF

• Click on Export as PDF on right side of gitbook

Contributing to documentation

The documentation is build using GitBook software from Markdown files in the .

We encourage you to participate in improving the documentation. From corrected typos to new sections, every commit is appreciated. You can access the source files by clicking the "Fix this page"-button in the GitBook or by selecting the relevant Markdown-file in the documentation. You can commit your changes by sending a pull request.

1. Fork the repo, do work in a feature branch.

2. Issue a pull request.

3. Make sure the automated test suite succeeds. They will show-up in the pull request.

4. Sign the CLA using .

5. Assign a maintainer or other developer on this topic to accept/evaluate your pull request. The current maintainer can be found in the .

Some information on GitBook and using Markdown can be found , more elaborate information on GitHub-flavored Markdown is found . If you like to upload illustration, you can use git or

If you are completely new to this and you need help to get started, file an issue in the documentation repository.

Chapters in the documentation

It may be obvious to you already from the index, however, here is an overview on what documentation goes in which chapter.

• Chapter 1 consist of an open smart grid platform introduction and architecture for potential users, architects and developers. The open smart grid platform website is used for basic product information.

• Chapter 2 contains the general userguide for open smart grid platform users

• Chapter 3 contains community related topics

• Chapter 4 contains information related to the open smart grid platform domains

• Chapter 5 contains information related to the protocols and simulators

• Chapter 6 shows the support options

• Chapter 7 code and documentation license

Versioning within the documentation

We have chosen to work with GitBook since it allows us to make different versions of documentation for each release. This is done by branching the files in the documentation repository. The master branch is used for releases only. The development branch is our main and current branch. If you like to improve the documentation, start a feature branch with your changes and send a pull request to the development branch.

Guidelines for new documentation

• The master branch is only used by major open smart grid platform releases

• Don't commit directly to the development branch, please do a pull request.

• We use the American spelling

• Please follow the used chapter and section numbering and apply it in your commits as well

• Currently we do not use image numbering, since it is too much of a hassle to keep it up-to-date. If you have a smart idea to do this, let us know!

• Give your (sub) document a relevant name or section with number

• Update SUMMARY.md if needed

Documentation inspiration

CDMA throttling

The main config is enabling the throttling client in your application

throttling.client.enabled=true

The throttling client needs to know where to find the throttling service and tell the service what threshold to use. This is configured using the following settings:

throttling.configuration.name=CDMA
throttling.configuration.max.concurrency=1000
throttling.service.url=http://localhost:9090
throttling.service.timeout=PT30S

Multiple clients can use the same configuration by using the same throttling.configuration.name. Throttling will be globally handled across those clients. Make sure all clients that use the same name also use the same throttling.configuration.max.concurrency. The last client to register a configuration will determine the max concurrency (overwriting previous registrations).

The throttling service itself should be deployed only once, it is not suited for load balanced deployment.

Sys Admins who are tasked with keeping the Open Source Grid Platform running on a environment, can find some information in this chapter.

Get Started

To get started with Open Source Grid Platform, please read our Introduction. The Introduction offers an excellent overview of the Platform and its features.

A next step could be to have a look at the WSDL's to understand which functions are present per functional domain. Depending on the functional domain one is interested in, one could also have a look at the Protocol Adapter and device simulator for the domain.

Installation

If a full installation is desired, have a look at our Installation Guide. This can be used to setup a development environment which can be used to start the Platform and run it. Installation on one or several servers can be derived from the steps within the Installation Script.

All the features of the open smart grid platform are accessible by it's webservices, as defined in the WSDL files. To use these services to communicate with devices through the platform, please keep in mind the following things.

• A Soap request will have to be generated from your Application.

• In the WSDL and xsd files of the Platform the Requests and it's objects are defined.

• Fill in the parameters once you have an empty soap request for a certain function. The restrictions/requirements are defined in the WSDL files.

• The request should have a header where an ApplicationName, UserName and OrganisationId are defined. At the moment User and App name are not used in the Platform (Except for audability and logging). The OrganisationId, however, must match a known organisation within the Platform. This organisation must have the right authortities to make a certain request.

• Furthermore, the Request must be signed with a certificate from the Organization with the OrganisationId in the header.

• The requests use the Secure HTTP protocol (https).

• For Asynchronious requests, the Platform will respond with a correlationId after succesfully making the request. Use this correlationId in the matching Response-Request to obtain the response from the device.

To learn more about the open smart grid platform's webservices, please take a look at the Domain Documentation or for hands-on experience with the Platform's webservices follow the UserGuide.

Rest

It is not possible to communicate with the Platform directly using REST webservices. As mentioned above, the open smart grid platform uses SOAP (For reasons defined here). If you want to use REST for your front end, you can write an Integration Layer that serves as a Soap Client and exposes the Soap calls through Rest web services that you can access from your front end.

Flows

When using the SOAP Web service, there are 2 flows that can occur:

• some calls are synchronous: a response is returned immediately;

• other calls are a-synchronous: an initial response contains a correlation id, which can be used to obtain the actual response.

Like the technical community as a whole, the open smart grid platform community is made up of a mixture of professionals and volunteers from all over the world, working on every aspect of the mission - including mentor-ship, teaching, and connecting people.

Diversity is one of our huge strengths, but it can also lead to communication issues and unhappiness. To that end, we have a few ground rules that we ask people to adhere to. This code applies equally to founders, mentors and those seeking help and guidance.

This isn’t an exhaustive list of things that you can’t do. Rather, take it in the spirit in which it’s intended - a guide to make it easier to enrich all of us and the technical communities in which we participate.

This code of conduct applies to all spaces managed by the GXF project (a LF energy project). This includes IRC, the mailing lists, the issue tracker, DSF events, and any other forums created by the project team which the community uses for communication. In addition, violations of this code outside these spaces may affect a person's ability to participate within them.

If you believe someone is violating the code of conduct, we ask that you report it by emailing the mailing list gxf@lists.lfenergy.org.

Be friendly and patient.

Be welcoming.

We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, color, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.

Be considerate.

Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions. Remember that we're a world-wide community, so you might not be communicating in someone else's primary language.

Be respectful.

Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It’s important to remember that a community where people feel uncomfortable or threatened is not a productive one. Members of the open smart grid platform community should be respectful when dealing with other members as well as with people outside the open smart grid platform community.

Be careful in the words that you choose.

We are a community of professionals, and we conduct ourselves professionally. Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behavior aren't acceptable. This includes, but is not limited to:

• Violent threats or language directed against another person.

• Discriminatory jokes and language.

• Posting sexually explicit or violent material.

• Posting (or threatening to post) other people's personally identifying information ("doxing").

• Personal insults, especially those using racist or sexist terms.

• Unwelcome sexual attention.

• Advocating for, or encouraging, any of the above behavior.

• Repeated harassment of others. In general, if someone asks you to stop, then stop.

When we disagree, try to understand why.

Disagreements, both social and technical, happen when people get passionate about what they are doing. It is important that we resolve disagreements and differing views constructively. Remember that we’re different. The strength of open smart grid platform comes from its varied community, people from a wide range of backgrounds. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint doesn’t mean that they’re wrong. Don’t forget that it is human to err and blaming each other doesn’t get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.

This Code of Conduct is based on the Django Code of Conduct

Installation

This document describes the automatic installation procedure for your Open Smart Grid Platform development environment.

Manual installation

If you would like to follow the manual installation procedure, please proceed to the Manual Installation Chapter.

Overview

Creating a Virtual Machine using Virtual Box and Vagrant

To improve the usability of the Installation process, a Vagrant file and some puppet scripts are used to automatically set-up an virtual Open Smart Grid Platform development environment. The following steps will describe how to install VirtualBox, Vagrant and kick off the procedure by running the vagrant up command.

System Requirements

The following system requirements are recommended:

• Core i5/i7 ~2.5GHz Dual Core, Quad Core recommended

• At least 16 GB RAM, 32 GB RAM recommended

• At least 20 GB free space, 50 GB free space recommended

The installation procedure has been tested on Windows 10.

Install Vagrant and VirtualBox

Start by downloading VirtualBox by going to https://www.virtualbox.org/wiki/Downloads And follow the installation steps.

note: If you already have VirtualBox, make sure it is at least version 6.1.38

note: Check whether Virtualbox stores the images on a drive with enough free space. (Open Oracle VM VirtualBox Manager -> Preferences -> General -> Default Machine Folder).

Now download and install Vagrant. Vagrant is available at the following URL: https://www.vagrantup.com/downloads.html

note: If you already have Vagrant, make sure it is at least version 2.3.0 Complete the installation and restart your PC.

note: If you did a fresh install of Vagrant and already had a command prompt open, make sure you close this command prompt and open it again.

Tip

• Remember to enable Virtualization in your system BIOS.

• Also disable Hyper-V in Windows (can be found in Windows Features)

Download and run the Vagrant file

First create a new directory (for example: D:\My Vagrant Images\OSGP Development\)

Browse to https://github.com/OSGP/Config/tree/development/vagrant and save the png image and Vagrantfile files in your newly created directory.

Note

• Make sure that the file is named like this: Vagrantfile without an extension!

• If the file has an extension (for example .txt) you can rename the file using the following console command.

  MacOS/Linux:

  Copy

  mv Vagrantfile.txt Vagrantfile

  Windows:

  Copy

  move Vagrantfile.txt Vagrantfile

Now open a Command Prompt and navigate to the newly created directory where you just put the files. Make sure that you run the Command Prompt using administrator rights.

note: When you open the Vagrantfile you see that default the image is configured to run in virtualbox with 2 cpu cores and 8192 MB of RAM. If you need to you can change this to more or less cpu cores and RAM, but it is recommended to use the provided settings.

Run the following command: vagrant up

note: In case of error bad uri Images/OSGP Development/hashicorp/itc/itc-ubuntu-mate-20.04-amd64 then use the following command;

• vagrant destroy

• vagrant box add itc/itc-ubuntu-mate-20.04-amd64

• vagrant up

note: In case of an error complaining about not being able to resolve a URL (for instance to github.com) then try using a different internet connection not behind a proxy.

Vagrant will now automatically download an Ubuntu image (+- 2.6 Gb), create a virtualbox image from it and run the installation puppet script when finished. This might take a while, depending on your internet speed. After some time (while the script is still running) you will notice that a window with an Ubuntu Virtual Machine pops-up. Don't log in yet, wait until the script in the Console is finished.

Tip

• If the script fails for some reason (eg. Errors in the console such as time outs during downloading), you can retry the procedure by running the following command vagrant destroy && vagrant up

Now that the script has ran its course, it will automatically log in on the Ubuntu virtual machine as user dev.

note: For some actions, like sudo, you will have to enter the password of user dev. The password for user dev is dev.

Optimize your Open Smart Grid Platform Development virtualbox image.

At this point you also can adjust the virtualbox settings like cpus and memory size. If you don't want to adjust this proceed to Chapter 2.1.2. Platform Setup.

If you do want to update the virtualbox settings for this image, shut down the image first:

Once the machine has been Shut Down, open VirtualBox and right click on the new virtual machine (called "OSGP Development") and select Settings. Go to System and increase the Base Memory of the system to at least 6144 MB (6 GB) (or the maximum recommended (in green) amount for your system).

Now go to the Processor Tab and increase the amount of Processors to the maximum recommended (in green) amount.

Close the Settings window and Start the Virtual Machine again. Once it is booted, you should be automatically logged in as the 'dev' user.

Post actions

In order to use git correctly you need to execute the following commands in a terminal:

git config --global user.name "your full name"
git config --global user.email yourEmailAddress

Recap

You just created a virtual machine running Ubuntu with pre-installed tooling. Proceed with Platform Setup of the guide describing how to set-up the open smart grid platform.

Adding an Organisation to the platform

This chapter describes how to add a new Organisation to the platform. This includes creating a certificate for the new organisation.

Creating an Organisation

In SoapUi go to DeviceManagement under the Admin project. Click on request 1 under CreateOrganisation. Fill in the parameters like the example request below. Make sure to sign the request with the test-org.pfx and to use the test-org organisation in the request header.

This request creates a new organisation called "my-org":

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:CreateOrganisationRequest>
         <ns1:Organisation>
            <!--type: Identification-->
            <ns1:OrganisationIdentification>my-org</ns1:OrganisationIdentification>
            <!--type: string-->
            <ns1:Name>my-org</ns1:Name>
            <!--type: string-->
            <ns1:Prefix>MYO</ns1:Prefix>
            <!--type: PlatformFunctionGroup - enumeration: [ADMIN,USER]-->
            <ns1:FunctionGroup>ADMIN</ns1:FunctionGroup>
            <!--type: boolean-->
            <ns1:Enabled>true</ns1:Enabled>
            <!--1 or more repetitions:-->
            <!--type: PlatformDomain - enumeration: [COMMON,PUBLIC_LIGHTING,TARIFF_SWITCHING]-->
            <ns1:Domains>COMMON</ns1:Domains>
            <ns1:Domains>PUBLIC_LIGHTING</ns1:Domains>
         </ns1:Organisation>
      </ns1:CreateOrganisationRequest>
   </soapenv:Body>
</soapenv:Envelope>

Authorise the new Organisation for the device

To use this new organisation to control the SSLD_000-00-01 device, the authorisations need to be updated. To do that the UpdateDeviceAuthorisations request will be used, it can be found under DeviceManagement in the admin project.

Fill in the parameters like shown below. The functionGroup will be set to AdHoc to authorise the 'my-org' organisation for the AdHoc functions.

Make sure to use the test-org as OrganisationIdentification in the request header, and to sign the request with the test-org.pfx.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/admin/devicemanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>AAPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>test-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:UpdateDeviceAuthorisationsRequest>
         <!--1 or more repetitions:-->
         <ns1:DeviceAuthorisations>
            <!--type: Identification-->
            <ns1:deviceIdentification>SSLD_000-00-01</ns1:deviceIdentification>
            <!--type: Identification-->
            <ns1:organisationIdentification>my-org</ns1:organisationIdentification>
            <!--type: DeviceFunctionGroup - enumeration: [OWNER,INSTALLATION,AD_HOC,MANAGEMENT,FIRMWARE,SCHEDULING,TARIFF_SCHEDULING,CONFIGURATION,MONITORING]-->
            <ns1:functionGroup>AD_HOC</ns1:functionGroup>
            <!--Optional:-->
            <!--type: boolean-->
         </ns1:DeviceAuthorisations>
      </ns1:UpdateDeviceAuthorisationsRequest>
   </soapenv:Body>
</soapenv:Envelope>

Creating a certificate for the new organisation

Now that the 'my-org' organisation is authorised to use the SSLD_000-00-01 device, it is time to create a certificate for the my-org organisation. This certificate will be used to sign the requests.

Open a terminal and navigate to /home/dev/Sources/OSGP/Config/certificates/

A script has been created to create the certificates, execute it by running the following command in the terminal:

./create_client_cert.sh my-org 1234 1234

You should receive similar output as shown in the screenshot below.

Now that the certificate has been created, restart the tomcat server.

Signing a request with the new certificate

When the tomcat server is up and running again, go to SoapUi and add the new certificate to the public-lighting project: double-click on the project, go to the WS-Security Configurations tab and select the keystores tab. Click the '+' button and browse to the my-org.pfx certificate which can be found in /home/dev/Sources/OSGP/Config/certificates/osgp-ca/certs/

Now double-click on 'Request 1' in SetLight in PublicLightingAdHocManagement in the public-lighting project. Set the SSL Keystore to 'my-org.pfx' in the request properties so the request gets signed with the new certificate. Change the request parameters as shown in the example below:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns="http://www.opensmartgridplatform.org/schemas/common/2014/10" xmlns:ns1="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10">
   <soapenv:Header>
      <ns:ApplicationName>APPLICATION_NAME</ns:ApplicationName>
      <ns:UserName>USER_NAME</ns:UserName>
      <ns:OrganisationIdentification>my-org</ns:OrganisationIdentification>
   </soapenv:Header>
   <soapenv:Body>
      <ns1:SetLightRequest>
         <!--type: Identification-->
         <ns1:DeviceIdentification>SSLD_000-00-01</ns1:DeviceIdentification>
         <!--1 to 6 repetitions:-->
         <ns1:LightValue>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:Index>0</ns1:Index>
            <!--type: boolean-->
            <ns1:On>true</ns1:On>
            <!--Optional:-->
            <!--anonymous type-->
            <ns1:DimValue>50</ns1:DimValue>
         </ns1:LightValue>
      </ns1:SetLightRequest>
   </soapenv:Body>
</soapenv:Envelope>

Note the OrganisationIdentification is now set to 'my-org'. Send the new request, you should receive the following response:

<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
   <SOAP-ENV:Header/>
   <SOAP-ENV:Body>
      <ns2:SetLightAsyncResponse xmlns:ns2="http://www.opensmartgridplatform.org/schemas/publiclighting/adhocmanagement/2014/10" xmlns:ns3="http://www.opensmartgridplatform.org/schemas/common/2014/10">
         <ns2:AsyncResponse>
            <ns3:CorrelationUid>my-org|||SSLD_000-00-01|||20160805150420802</ns3:CorrelationUid>
            <ns3:DeviceId>SSLD_000-00-01</ns3:DeviceId>
         </ns2:AsyncResponse>
      </ns2:SetLightAsyncResponse>
   </SOAP-ENV:Body>
</SOAP-ENV:Envelope>

Check the device-simulator to see if the dimValue of the SSLD_000-00-01 changed to 50.

You now have successfully created a new organisation, along with a certificate to sign the requests, and changed the device authorisations of the device to accept commands from the new organisation.

Developer tools, technical guidelines and continuous integration

Tools Used

The technology and tools used can be found in the section.

Code Guidelines and Code Tests

• To prevent formatting issues caused by using different IDEs, the formatting style for the code is based on with some extra additions based on the Eclipse based conventions that were already in use.

  Check the explanation of the as well as the instructions how to configure your idea to be able to format according to the conventions (available for and ).

• Follow approach for branching

• Write behaviour driven tests using , see the

• In case you are not familiar with behaviour driven tests, include unit tests

• Fix SonarQube issues

• Issue pull request (preferable to development branch)

Continuous Integration

All changes pushed to GitHub are built by our build server. Pull requests to master branch or development branch are also built. SonarQube performs static code analysis to help improve the quality level of the code base.

Technical Conventions and Rules for New Code

This project is engineered, built and tested using Domain Driven Design and Behaviour Driven Tests.

• Use the Frameworks, don't roll your own

• Single class, single responsibility

• Value objects are immutable

• Map generated objects to value objects or entities

• Transferring an object means using a DTO

• Use base classes for common logic

• Interfaces are good, but 'impl' is bad

• Extend classes, don't expand classes

• Migrate databases using Flyway

• JMS for messaging

• Extend the authorization logic if needed, don't bypass it

• Log errors/exceptions

• Add meaningful comments to the code

• Follow the code guidelines

Development Guide per Component

Web Service Adapters

The web service adapters use Spring Web Service, contract first. JAXB is used to generate Java classes from the XSD's. All SOAP operations are bound to an endpoint. The incoming SOAP requests are authenticated by organization identification (plus certificates). Organization authorizations are checked for the desired operation. If the request is OK, a JMS message is sent to a domain adapter component.

Domain Adapters

Domain adapters contain business logic and persistence. Domain adapters process and forward the JMS message to the Core component.

Domain Components

Domain components contain entities and value objects.

Core

The Core component routes messages from domain adapter components to protocol adapter components and vice versa. Furthermore, it offers read-only database access for protocol adapter components.

Protocol Adapters

Protocol Adapter components translate a message from domain adapter components into a protocol message for a smart device. Protocol Adapter components send the protocol message to a smart device using a network connection. The response from the smart device is translated into a domain response message which will be sent to the Core components (which will route it to the domain adapter which issued the request).

OSLP

For the OSLP implementation, two components are used. The first component is the protocol adapter for the protocol. It can translate message into the protocol message for SSLD's. Second there's the signing-server component. It is responsible for signing the protocol message using the private key of the platform. The components communicate using a key-pair. The signing-server can handle multiple protocol adapter instances by utilizing a reply-to queue per protocol adapter instance. Since the protocol adapter component needs to be reachable from a network, it is a requirement that the private key may not be used by the protocol adapter directly. The signing server component can be deployed in such a way that no network access is available to this component, as the only coupling needed are the queues / the message broker.

Other Guides

Installation Guide

Create New Domain Guide

Thank you for contributing to the Open Smart Grid Platform. Please keep the following in mind before submitting code.

Guidelines

Before code is merged it needs to comply with a number of guidelines: 1. Code should be as complete as possible (preferably no placeholders, TODO's or FIXME's) 2. Right formatting; code should follow the Coding Conventions (see 3.1.2) 3. Fixed/added unit tests where applicable 4. Javadocs added where applicable 5. Accepting pull-requests with SonarQube reports "Blocker" and "Critical" are not allowed 6. Comply with International open standards where possible (e.g. IEC standards)

Contributor License Agreement

We ask each of our contributors to sign our contributor license agreement (CLA). This has advantages for both parties, it gives you the assurance that your contribution will remain available under the Apache 2.0 license. Meanwhile, you give your code in license to us, so we can add your code to the open smart grid platform. And we know your contribution is entirely your work, so we don't get in trouble with legal issues. Please read the CLA text carefully.

Submitting code

To submit changes to the open smart grid platform branches: 1. Create a fork of the open smart grid platform repo you will be working in 2. Make and commit your changes to your fork 3. Create a pull request to merge the changes into the right branch (see 4.1.4 for the branching strategy) If the changes fix a bug, mention the issue number in the commit message or pull request message (example: fixes 101, solved 87). If no ticket exists, create one beforehand. Afterwards, please update the relevant documentation in this GitBook.

Open Source Branching Strategy

The open smart grid platform's main branch is master. All major releases are tagged in this branch. Development is done in the development and feature branches. We use the GitFlow branching strategy. Find more information on this strategy here:

The GitFlow workflow is someone complicated, but it has the advantage that it gives a clear overview of all previous releases and current development and thus helps to collaborate more efficiently. Please follow this strategy in your commits.

Pull requests: review process

Anyone can send in a pull request. Assign a maintainer or other developer with knowledge on this topic to accept/evaluate your pull request. You can view the SonarCube test results at () and the Jenkins continuous integration results at ()

If your code is a useful contribution and meets our quality standards (see section 3.1), it will be added to the open smart grid platform! Developers are in charge of judging this. Don't forget to update the documentation as well.

Grid eXchange Fabric (GXF) is a project of LF Energy. LF Energy is part of the Linux foundation.

Chapter 4 Domain This chapter describes the separate domain in the open smart grid platform.

Web Service Adapters

The web service adapters use Spring Web Service, contract first. JAXB is used to generate Java classes from the XSD's. All SOAP operations are bound to an endpoint. The incoming SOAP requests are authenticated by organization identification (plus certificates). Organization authorizations are checked for the desired operation. If the request is OK, a JMS message is sent to a domain adapter component.

Domain Adapters

Domain adapters contain business logic and persistence. Domain adapters process and forward the JMS message to the Core component.

Domain Components

Domain components contain entities and value objects.

We invite everyone to participate in the development of the open smart grid platform. There are multiple ways to support the project.

• submit issues about bugs or requested features

• solve issues

• develop new features

• write or improve the documentation

This chapter contains all the non-technical and general technical knowledge for developers to start contributing.

If you want to get in touch to discuss non-technical subjects, send us an email to the LF energy GXF mailing list gxf@lists.lfenergy.org or open an issue on Github.

New Features

1. If there is a need (or wish!) for a new feature, add it as issue to Github. Please provide a full description about the background of the problem.

2. A developer can take on the issue and start working on it on voluntary base. If you need this feature and you have the money to pay for it, you can hire a developer and have the developer work on it. If open smart grid platform core components are involved, please discuss your change first with one of the developers/maintainers.

3. The developer makes a description on how he wants to fix the problem. Other developers can discuss the solution as well. If everybody agrees on the solution direction, the developer codes the solution and submits it (by sending in a pull request). The developer should also document the feature in the GitBook

4. The maintainer can check the code (or assign this to someone else) and merge it with upstream releases.

Bug tracking

1. Find out as much as possible about the bug before reporting it. Please check on GitHub/Jira whether the bug has already been reported. Also, look for logs, error messages etc. Please include as much information as possible background on the bug and submit the bug on Github or Jira.

2. The maintainer makes sure that somebody will look at the bug, check for duplicates and thank the contributor for sending in the bug. If the bug turns out to be a duplicate, the issue will be closed.

3. A developer will try to reproduce the bug and will look for the root cause. The developer adds his findings to the issue. If the developer cannot reproduce the bug, the developer will contact the user for more information or/and login into the user's system (when possible for the user/developer). If it's impossible to re-produce the bug, the issue will be closed.

4. Otherwise, the developer will write a patch and tests the fix.

5. Once the patch is accepted (see Code review/test process), it will be shipped with the next release.

6. The maintainer than closes the issue.

Questions

If you have a question, please create an Github issue in the documentation repository.

Report security issues

Due to the sensible nature of security issues e.g. zero days, we prefer a responsible disclosure. Security issues can be reported to gxf+owner@lists.lfenergy.org.

• How to start everything up? Make sure that PostgreSQL is running. Make sure that Apache HTTPD web server and Apache ActiveMQ are running. Then start Apache Tomcat application server as described in the installation manual.

• Where are the log files? The Apache Tomcat application server logs can be found in /var/log/tomcat. The Apache HTTPD web server logs can be found in /var/log/httpd. The PostgreSQL log files can be found in /var/lib/pgsql/9.3/data/pg_log. The platform log files can be found in /var/log/osp/logs/.

• Help: SELinux is preventing < something >? Make sure to set SELinux to 'permissive' mode. Then try again and it should work as SELinux will no longer enforce the current policy. Later, one can use the SELinux tools to create a proper policy that allows everything that was prevented before.

• How to add or update a component? Make sure to place the properties file(s) for the component in /etc/osp. Add the locations of the properties file(s) to Apache Tomcat context.xml file. Add the war file to /usr/share/tomcat/webapps. Restart Apache Tomcat.

• How to configure a component? Most (if not all) components of the open smart grid platform are de-coupled using queues. Configure the broker URL in the properties file and take notice of the queues that a component uses/needs. Make sure to double check the connectionstring for PostgreSQL.

• How to configure URL's for a component? In this case the Apache HTTPD vhost needs to be updated. The vhost config file can be found in /etc/httpd/conf.d. We use redirects from HTTP to HTTPS and AJP proxy to send the requests to Apache Tomcat.

• How to check up on Apache ActiveMQ? Apache ActiveMQ offers a web interface (the default port is 8161, default credentials: admin/admin). Using the web interface one can check the queues and especially the dead letter queue (DLQ).

• How do I obtain a public key for a device? Public Keys are usually manufacterer supplied for Smart Metering. For the Open Source part you can use the OSLP device simulator. In Sources/OSGP/Config/certificates/oslp/ you can find instructions for generating a OSLP device public key, and a folder with 5 pre-generated keys for test devices.

• My code gives a lot of errors in Eclipse after importing Try the following things: run mvn clean install in the open-smart-grid-platform directory. Right-click on a project in Eclipse and select 'Maven -> Update project..', select all projects and update.

• The Vagrant script fails If you are receiving errors while downloading the ubuntu iso, sources, etc. or if the puppet script does not start; try running the Vagrant script again by typing vagrant destroy && vagrant up in the directory with the vagrant file.

• I want to update my code from Github If you want to update your code, just run git pull in the repository you want to update. You can also create a fresh Virtual Machine using the vagrant installation, this procedure makes sure you have the most recent code.

• Is a user required to consume platform services? No, an organization is required.

_If your question is not in this list, please create an [issue on Github in the documentation repository ]documentation repository

Deployment options

Hosting the open smart grid platform in the cloud is possible, as well as on premises.

Active-active setup over multiple datacenters

If you like to setup an active-active installation over multiple datacenters. Make sure that the open smart grid platform database runs redundant over both datacenters (master-slave configuration).

Maintenance

There's not much maintenance that needs to be performed. Archiving some old log files, checking up on available disk space and creating a backup of the databases. Looking into the queues to see if there are no messages in the dead letter queue.

With the open smart grid platform we intend to have the right balance between a benevolent Dictator and a Formal Meritocracy in order to have a balanced decision-making process and to prevent unwanted dictators and everlasting discussions. The basic principle is that decisions are based on consensus. If this decision making process takes too long or a decision is required, the community council has the authority to make a decision. Grid eXchange Fabric complies with the LF Energy governance

The governance and rules should be respected.

Community council / Technical Steering Committee

The community council consists of 5 members and has the authority to make decisions on all community related subjects.

The community council is responsible for:

• General ambitions, objectives and goals

• Principles and understandings

• Governance and consultation bodies

• Guidelines and procedures and tool selection

• Contribution (process) of individual members

• Architectural and (development) infrastructure choices

• Raise subjects/issues that are important for the direction/development of the community

When the community grows, members of the community council can be elected. If the situation demands or requires it, the community council has the ability to establish sub councils for a specific subject, area of domain.

The community council consist of the following members:

• Robert Tusveld - Lead Architect - Chairman

• Maarten Mulder - Product owner GXF & LF Energy TAC member for GXF

• Paul Houtman - Lead Architect

• Kevin Smeets - Maintainer

• Jonas van den Bogaard - OSPO team member & LF Energy TAC member for Alliander

If you would like to join the community council, please contact us! Mention @OSGP/communitycouncil in an github issue. The (online) community council meetings will happen when needed.

Maintainers

Maintainers are responsible for maintaining parts of the code-base. Maintainers have the following responsibilities

• Coordinate development activity

• Make sure code/documentation reviews are being done

• Coordinate pull-requests

• Coordinate bug follow-ups

• Coordinate questions

In case of long discussions or arguments, maintainers or other can request a community council decision.

Current maintainers:

• GXF Platform and smart lighting domain: Kevin Smeets

• GXF Platform and Smart metering domain: Stefan Ermens

• GXF Architecture & documentation: Robert Tusveld

Contributors

Contributors include anyone in the technical community that contributes code, documentation, or other technical artifacts to the project.

Anyone can become a contributor. There is no expectation of commitment to the project, no specific skill requirements and no selection process. To become a contributor, a community member simply has to perform one or more actions that are beneficial to the project.

This domain covers the use of the open smart grid platform for smart lighting.

Scope

The goal of this domain is to control, monitor and manage (street) lights at scale. It allows streetlight owners to control/manage the (street) lights in an more intelligent way compared to ripple control technology.

Features

This domain consist of , groups, light sensors, manual switching and monitoring of a typical public lighting use-case.

PublicLighting webservices

The open smart grid platform core and admin functions for device management.

Scope

This core and admin domain contains all generic web services that can be used in other domains. Most generic services relate to device management including powerful device authorization management

Common webservices

• Core WSDL's

• Core XSD schema's

Admin webservices

• Admin WSDL's

• Admin XSD schema's

This is a guide to start contributing to the open smart grid platform project:

1. Make yourself comfortable with the open smart grid platform by e.g. installing it.

2. Read the documentation to get an idea of how the software works (architecture chapter), how the community works (this chapter).

3. Find an open issue to work on or fire an new issue.

4. Assign yourself to the issue and start contributing.

5. Start contributing by using the procedures mentioned in this chapter.

Schedules for light switching can be set using Set Schedule requests from the Public Lighting Schedule Management web service. For brevity the XML element and type names in the descriptions below will not include the namespace (which will typically be "http://www.opensmartgridplatform.org/schemas/publiclighting/schedulemanagement/2014/10").

A switching schedule is defined by a number of declarations of switching moments (also known as schedule entries). The SetScheduleRequest defines the schedule, where Schedules of type Schedule define the entries. A complete schedule for a device as set with the Set Schedule request can have 1 up to 50 entries. Each schedule entry defines a moment on a day when certain relays on a device are switched on or off. Whether or not a switch action defined in a schedule entry is executed may not only depend on the entry itself. Other switch moments from the schedule that are close in time compared to an entry may cause switching to be skipped.

A more detailed description of the components defining a schedule entry is in the sections below:

• week day

• time

  • explicitly configured fixed time

  • time of sunrise or sunset

  • time of sunrise or sunset combined with sensor input

• light value

Week Day

The value of WeekDay is used to indicate on which days the schedule entry may trigger switch actions.

Time

Each schedule entry can cause switching at a single time during the day. There are a number of ways in which this time can be specified, starting with ActionTime.

For ActionTime values SUNRISE or SUNSET the value of TriggerType specifies what the actual switching time should be.

Fixed Time

For ActionTime ABSOLUTETIME a fixed time can be set for the switching moment as Time. The Time value needs to be formatted in a way the protocol implementations can handle. For the currently listed implementations, you should be fine when you use a format from:

• hh:mm;

• hh:mm:ss;

• hh:mm:ss.SSS

With hh from 00 to 23, mm from 00 to 59, ss from 00 to 59 and SSS from 000 to 999. Some protocols may accept more precise time formats than they support. The IEC61850 implementation for instance, will silently apply only the hours and minutes from any of the formats listed above.

Astronomical Time

For ActionTime SUNRISE or SUNSET with TriggerType ASTRONOMICAL the astronomical sunrise or sunset time (as calculated by the switching device, based on its longitude and latitude) will be used to determine the switching moment.

Astronomical Offsets

If an astronomical offset is configured, it has to be added to the calculated astronomical time to determine the time to be used as the switching moment. For positive offset values, the astronomical time for the switching moment will be the configured amount of minutes after the calculated astronomical sunrise or sunset time, while for negative values the astronomical time used will be the number of minutes before the calculated astronomical sunrise or sunset time.

####Astronomical Sunrise Offset

The astronomical sunrise offset is applied with entries with ActionTime SUNRISE and TriggerType ASTRONOMICAL. The following picture is an example of switching off at 07:30; the calculated astronomical sunrise (say at 07:15 for the day shown) plus 15 minutes (configured as AstronomicalSunriseOffset 15).

####Astronomical Sunset Offset

The astronomical sunset offset is similar to the astronomical sunrise offset, except that it is applied with entries with ActionTime SUNSET and TriggerType ASTRONOMICAL.

Astronomical Time With Sensor

For ActionTime SUNRISE or SUNSET with TriggerType LIGHT_TRIGGER the calculated astronomical sunrise or sunset time will be used as a reference time with a trigger window to determine the switching moment. The astronomical time itself is calculated in the same way as with astronomical time (without light sensor input). Switching happens within a configured trigger window around the astronomical time, at a moment that is influenced by a signal from a light sensor.

Trigger Window

The TriggerWindow with its minutesBefore and minutesAfter defines a window of time around an astronomical sunrise or sunset time with sensor. Switching will occur at the start of the window when light sensor input is received before the window. Switching will occur at the end of the window when light sensor input is not received before the end of the window. Switching will occur at the time light sensor input is received, when this input is received within the window.

Light sensor input in the conditions above means the sensor trigger for light when the schedule entry is switching off, and the sensor trigger for dark when switching on.

Astronomical Time With Sensor Signal Within The Trigger Window

The following picture is an example where the light sensor reports dark within the trigger window for a schedule entry for astronomical time with sensor signal. Switching on occurs at the time the sensor input is received. Note that for this example this could have been at any time between 19:45 and 20:30 (15 minutes before to 30 minutes after the astronomical sunset, calculated to occur at 20:00 on the day shown).

Astronomical Time With Sensor Signal Before The Trigger Window Opens

The following picture is an example where the device has received a light sensor report before the trigger window opens for a schedule entry for astronomical time with sensor signal. Switching off occurs at the start of the trigger window.

Astronomical Time Without Sensor Signal Before The Trigger Window Closes

The following picture is an example where the device has not received a light sensor report before the trigger window closes for a schedule entry for astronomical time with sensor signal. Switching on occurs at the end of the trigger window.

Minimal Burning Time

For certain types of lighting it may be undesirable to switch the lights on only for a short period of time, after which they are switched off again. In such a case the action of switching the lights on will be suppressed if minimumLightsOn is set with a positive number of seconds, and the action switching the lights off again is expected within this time period.

The minimal burning time is always regarded with respect to an actual time for a switching moment that switches a relay on in comparison with the expected time of the next switching moment where the same relay will be switched off again. Switching on will be skipped if switching off is expected to occur within a number of minutes set as minimumLightsOn with the schedule entry that switches the relay on.

Minimal Burning Time With Astronomical Offset

This example shows the minimal burning time preventing the morning lights to be switched on at a fixed time because switching off at the calculated time of astronomical sunrise (with offset) would happen before passing of the minimum number of minutes the lights should be kept on.

Minimal Burning Time With Light Sensor Trigger Window

This example shows the minimal burning time preventing the morning lights to be switched on at a fixed time because switching off at the start of the trigger window around the calculated time of astronomical sunrise would happen before passing of the minimum number of minutes the lights should be kept on.

Light Value

Each schedule entry may include 1 to 6 LightValue elements. These light values determine the relay to switch, whether the relay should be switched on or off, and whether the lights with a relay should be dimmed (and by how much).

• Index: 0 for all light switching relays in the device, or 1 to 6 for numbered relays (the

  index should indicate an existing relay that is used for light switching).

• On: true if this entry is for switching on the relay(s) identified by Index; false for

  switching off.

• DimValue: optional percentage set as number 1 to 100 indicating a dim value; will be ignored

  when the protocol or switching device does not support dimming.

Common Light Scheduling Patterns

Here are some examples of patterns that are common with light schedules. The patterns are formed by combinations of schedule entries that switch on or off lights controlled by a certain relay on the switching device.

All Night Lights

All night lights is a name for lights that are turned on around sunset and keep burning all night until they are switched off again around sunrise. The all night lights are switched by a pair of schedule entries:

• one entry switching on at astronomical sunset time with

  optional offset or light sensor trigger.

• another entry switching off again based on astronomical sunrise time with

  optional offset or light sensor trigger.

Morning Lights

Morning lights is a name used for lights that are switched on a short period in the morning hours of a day to illuminate a period before or around the morning twilight. The morning lights are switched by a pair of schedule entries:

• one entry switching on at a fixed time;

• another entry switching off again based on astronomical sunrise time with

  optional offset or light sensor trigger.

Fixed Time And Sunrise Interaction

Depending on the location of the device the time of sunrise may vary quite a bit throughout the year. Because of this it is possible that what for some period would be a very reasonable schedule, is a questionable schedule (possibly to be considered invalid) in another season.

During the summer in the Netherlands for example sunrise can be as early as approximately 05:15, while during the winter the sun may rise even a little later than 08:45. For this example we will assume configuration for the morning lights to switch on at fixed time of say 06:00. This is a time after the earliest sunrise in the year, but well before the latest sunrise in the year. To complete the morning lights configuration, a second switching moment is configured to switch the lights off at sunrise. With this set up the lights will be switched off after having been on for almost three hours at some time in the winter (for instance from 06:00 to 08:45). During summer at some days the lights will not be switched off in the morning at all because they were turned on (at 06:00) after sunrise (switching off at any time before 06:00, for instance at 05:30).

Whether the lights stay on all day in the summer or not is something to be looked into. A switching device may have logic to deal with this situation figuring out the switch off belongs with the later switching moment to turn the lights on, and decide not to switch on. If not, some validation may be needed to enforce such schedules not to be configured.

Evening Lights

Evening lights is a name used for lights that are switched on a short period in the evening hours of a day to illuminate a period after or around dusk. This is similar to the morning lights, but in the evening instead of the morning, and the fixed time moment comes (normally) after the switch action around sunset.

The evening lights are switched by a pair of schedule entries:

• one entry switching on at astronomical sunset time with

  optional offset or light sensor trigger.

• another entry switching off again based on fixed time;

Fixed Time And Sunset Interaction

Depending on the location of the device the time of sunset may vary quite a bit throughout the year. Because of this it is possible that what for some period would be a very reasonable schedule, is a questionable schedule (possibly to be considered invalid) in another season. See the explanation around sunrise for a graphical example.

Evening/Morning Lights

A combination of morning lights and evening lights can be configured for a relay if the lights may be turned off for a period in the late night and early morning, as opposed to the all night lights that keep on burning all through the night.

Validity

The GXF Public Lighting Schedule Management web service does not do much validation, other than checking authorizations for the device identified by the DeviceIdentification from the SetScheduleRequest and whether the request conforms to its XML schema definitions.

If for certain applications more constraints are desirable, it is left up to those applications to make sure the requests made to the platform conform to those additional constraints. Examples of such constraints, that are not enforced by GXF, could be:

• no duplicated schedule entries;

• no schedule entries canceling the switch actions of other entries within some time window;

• schedule entries may be required for all days of the week;

• switching on and off might be required to happen each day in equal number of times and alternately;

• checking expected actions around daylight saving change;

• checking expected switching actions for days with the longest or shortest number of hours of daylight;

• light value indexes map to existing light relays on the device the schedule is set on;

• constraints from applying the provided input with specific devices or protocols.

Protocol Implementations For Light Schedules

• IEC61850

• OSLP v0.6.1