Skip to end of metadata
Go to start of metadata
  •  

High Level Functional Overview and Design Specifications

Problem Statement

Support remote replication for Southbound SDK based drivers. CoprHD should provide required APIs, data model and runtime to support remote replication for storage systems managed by SB SDK drivers.

All device specific replication features should be handled by drivers.  Different array types have different configuration of their remote replication implementation. The SDK object model, SDK APIs  and internal  CoprHD support for SDK API have to be flexible enough to handle the following device dependent features in generic way:

  • different remote replication configurations (1-to-1, 1-to-n, n-to-1, and others)
  • different granularity of replication link management operations: lun based, replication group based, replication set based
  • device specific replication modes: sync/async/periodic replication modes and possibly many other device specific replication modes for supported replication granularity
  • as part of provisioning of remotely replicated volumes and in remote link management operations, provide to client ways to specify device specific data in the request and pass this data to driver.
  • replication link management operations for all supported replication link granularities — establish/split, suspend/resume, failover/failback, sync, swap 

Remote Replication Configurations

         Analysis of remote replication support was done for the following list of arrays:

  • HP 3PAR (Remote Copy)
  • DELL (Remote Replication for DELL PowerVault MD arrays)
  • Hitachi (TrueCopy)
  • IBM (XIV Remote Mirroring)

        There are the following common flavors:

  • Remotely replicated pairs of volumes are outside of remote replication group container. In this case replication links between volumes are managed individually.
  • Remotely replicated pairs of volumes are inside remote replication group container. In this case there are two cases how remote replication links between volumes are managed:
    • Remote replication group enforces consistency of remote replication operations. In this case link management can be done only at group level. Link operation applied to any pair of volumes in the group is automatically applied to all pairs in the group. 
       
    • Even when remotely replicated pairs of volumes are inside remote replication groups, array allows link operations on individual pairs. Pairs are in a group, but operational group consistency can be either enforced or not. If operational consistency is not enforced, link operations applied to individual pairs do not affect other pairs in the group. For example, VMAX allows operations on individual pairs of volumes in RDF groups, when replication mode is sync. 

In the list of analyzed remote replication technologies, some arrays support both flavors, some arrays support only pairs inside remote replication groups. 

 

Examples:

remote replication operations for HP3PAR remote replication groups

remote replication operations for DELL Compellent individual volumes and write order replication consistency groups

remote replication operations for IBM XIV individual volumes and remote mirror enabled consistency groups

remote replication operations for HDS VSP individual volumes and true copy consistency groups

Design Approach / High Level Implementation Details

Definitions

Replication Mode — device specific mode of replication link between volumes: synchronous/asynchronous/asynchronous with write order consistency/periodic replication, etc.

Remote Replication Pair (RRP) — two remote replication elements in source-target relationship; attributes –- replication mode, replication state

Remote Replication Group (RRG) — discoverable device level container for remote replication pairs. Attributes:

  • source system, target system
  • replication mode
  • is group consistency for link operations enforced (true/false)
  • replication state
  • replication operation granularity for this group: group and/or pair, or none (when driver supports link operations only on replication set level)

Maps to Primary RemoteCopyGroup (hp3par), Remote Mirror Enabled CG (XIV), TrueCopy CG (HDS), Dell Write Order Replication CG. In these referenced arrays, replication group always enforces link operation consistency --- remote link management operations for remote replication pairs in the group affect all pairs in the group.

Note: We assume that remote replication groups are independent containers of volumes on device. Membership of volumes in remote replication groups affects only remote replication operations on these volumes. It does not impact membership of volumes  in any other volume containers on device or local operations on these volumes.

Remote Replication Set (RRS) — discoverable interconnected set of storage systems, managed by the same driver type, defined and configured for remote replication outside of CoprHD. Each system in a replication set has assigned replication roles: source, target or both. Replication sets are containers for replication pairs (RRPs) and replication groups (RRGs). Replication sets should have the following configuration data available at discovery time:

  • supported replication modes: replication set can support more than one replication mode for replication pairs provisioned in this set
  • replication modes for which group consistency of link operations is automatically enforced
  • replication modes for which group consistency of link operations cannot be enforced
  • types of replication elements which can be provisioned in this set: remote replication pairs (dell, xiv, hds), remote replication groups (hp3par, dell, xiv, hds) or both (dell, xiv, hds)
  • set of supported granularity of remote replication operations for elements provisioned in this set: replication set, replication group (hp3par, dell, xiv, hds), replication pair (xiv, dell, hds)
  • current replication state (if replication operations are supported on the replication set level)

In VMAX case, VMAX RDF director group is example of RemoteReplicationGroup .  

RRS and RRGs are considered part of discoverable storage configuration domain managed by drivers. 

FAQs

Q. Why do we need to have both RRSs and RRGs?

A. RRS is connectivity domain of storage devices enabled for remote mirroring. RRG is a container for RRPs and it corresponds to physical mirroring container on device which can guarantee consistency of link operations for all pairs in the group. In case when array supports pairs outside of group container or does not support RRGs, we use RRSs directly to provision remote replication pairs.  


Q. Is provisioning of remote replication groups supported?

A. Yes, we will support provisioning of remote replication groups.

Note: When user provision remotely replicated pairs in replication set which supports remotely replicated groups and remotely replicated pairs, and the user does not specify replication group, we will provision new replication pair for volumes directly in the replication set.

 Example of Remote Replication Set which supports groups and pairs elements:


Detailed discovery  of remote replication configuration supported by drivers (RRSs and RRGS) is required in order to understand available provisioning options for remotely replicated volumes, validate provisioning requests and be aware of impact of link operations on replication pairs to other replication pairs.

For example:

  1. If driver supports only replication groups in a given RRS, request to create remotely protected volumes in this RRS must contain existing replication group where the volumes should be provisioned.
  2. If driver supports group and pair elements in a given RRS, request to provision replicated volumes in this RSS without specified replication group in the request should result in provisioning of a new replication pairs directly in the RSS.
  3. If driver supports group elements in a given RRS, but replication mode specified in the virtual pool cannot enforce group consistency of link operations, than request to create replication pairs with group consistency should fail validation.
  4. If driver supports only pair elements in RRS, than volume provisioning request in a group should fail validation.
  5. If replication group supports only group granularity for link management operations, any link management operation on individual volume pairs in the system should translate to operation on all pairs in the group.  As the result of the operation, state of replication group and state of all replication pairs in the group should be updated accordingly in the system.
  6. When replication group has group consistency enforced, any link management operation on individual volume pairs in the system should translate to operation on all pairs in the group.  As the result of the operation, state of replication group and state of all replication pairs in the group should be updated accordingly in the system.
  7. When driver creates a new replication group for volumes, with required group consistency, only group granularity for link operations should be supported in this group.  
  8. When driver creates a new replication group for volumes, without group consistency required and with replication mode, which does not have automatic enforcement of group consistency, both, group and pair granularity of link operations can be supported for this group (depending on RRS capability for this).


Support for Device Specific Data

We will support ability for client to specify device specific data in remote replication provisioning requests and in remote replication link operations. This data will be handled as opaque name-value pairs in the CoprHD and will be used as "pass-through"  to driver. Driver is supposed to know semantics of this data and it should know how to process this data in the context of the given request. CoprHD does not discover this data and it does not use it in placement decisions.

Typical Use Cases

 This is subset of typical use cases for remotely replicated volumes.

  1. Provision remotely replicated volume/volumes directly in the selected remote replication set.
  2. Provision remotely replicated volume/volumes in the selected remote replication set and in a new replication group with/or without requirement for link management group consistency 
  3. Provision remotely replicated volume/volumes in the selected remote replication set and in the selected existing remote replication group. Property for link management group consistency will be inherited from group settings.
  4. Move remotely replicated volume/volumes from one replication group to another replication group in the same replication set.
  5. Establish remote replication for the specified replication pair in split state.
  6. Establish remote replication for the specified replication group in split state.
  7. Suspend remote replication for the specified replication pair in active state.
  8. Suspend remote replication for the specified replication group in active state.
  9. Resume remote replication for the pair/group in suspended state.
  10. Fail over pair/group in active state.
  11. Fail back pair/group in failed over state.
  12. Split remote replication for pair/group in active/suspended/swaped or failed over state.   
  13. Delete remote replication pair.

Resource Ownership

Remote replication sets are public — any tenant can use remote replication sets for provisioning.

Remote replication groups by default are public. By default, any tenant can use remote replication groups for provisioning of remote replication volumes.

System or security admin can assign ACLs to remote replication groups to restrict access to specific tenants. ACL permission: USE .

Remote replication pairs inherit ACLs of the project where pair source and target volumes are provisioned.

Remote Replication Support in CoprHD Components

This section describes high level details for remote replication support in system functional layers. 

What is required to support remote replication in CoprHD:

Discovery (ExternalDeviceCommunicationInterface.java):

  • for each storage system type, managed by SDK driver,  discover RRSs with their properties;
  • for each storage system, managed by SDK driver, discover remote replication groups where this system is either source or target;

Discovery data, including properties of remote replication sets and groups, is used to validate provisioning requests and to place remotely replicated elements in required RRS and when applicable in required RRG.

Based on discovered properties of remote replication sets and discovered properties of remote replication groups, the remote replication subsystem learn remote replication capability of each device type under management. 

Cassandra Data Model:

  • defines persistent types to store information about  discovered RRSs and RRGs, provisioned RRGs, RRPs;
  • adds "Remote Replication" protection to VirtualPool model class.
    • NOTE:  This feature will be affected by vpool simplification going forward. Storage topology defined in "Remote Replication" protection option will be moved to "template" object and will be used at provisioning time. 

Api Service:

  • define api service class RemoteReplicationManagementService to provide REST api for the following operations: 
    • move remote replication pairs between remote replication groups in the same remote replication set (remote replication group load balancing);
    • manage remote replication links between remote replication elements (pairs, groups, sets) including link management for remote replication pairs with volumes in consistency groups;
      • Supported link management operations: failover/failback, establish/split, suspend/resume, swap;
  • Define API Service classes to provide query REST methods for for remote replication pairs, groups, sets and consistency groups; 
    • For example: get all remote replication pairs for a given remote replication groups; get all remote replication groups for a given remote replication set, etc. 
  • add RemoteReplicationProtectionParam to virtual pool create request;
  • add new matcher class to AttributeMatcherFramework to support matching storage pools which can be used for remotely replicated elements;
  • add RemoteReplicationParam to createVolume REST request;

UI:

  • allow user to specify data for RemoteReplicationProtectionParam in "Remote Replication" option for "Data Protection" drop down in "Create Block Virtual Pool" dialog;
  • allow user to specify data for RemoteReplicationParam in "Create Block Volume" dialog; 
  • support new catalog service for Remote Replication Link Management Operations;  

Remote Replication Scheduler:

  • implement scheduling and placement of remotely replicated volumes based on virtual array, virtual pool, RRS and RRG

Remote Replication Api Service Impl:

  • process placement recommendations and build volume descriptors

BlockOrchestrationDeviceController:

  • add support to orchestrate workflow step to create remote replication link between remotely replicated volumes to RemoteReplicationDeviceController.

Remote Replication Device Controller:

  • create remote replication link for source and target volumes;
  • implement remote replication link operations;
  • move remote replication pairs between groups;
  • delete remotely replicated volume pairs.

ExternalBlockStorageDevce:

  • implement device level support for remote replication operations: translate controller calls to calls to device driver.

Storage Driver SDK: 

  • define model types for remote replication sets, groups, pairs and remote replication modes;
  • define discovery api methods for remote replication configuration; 
  • define remote replication driver methods: create remote replication groups and pairs, move pairs between groups, delete pairs, replication link management methods;

Detailed Description

Below we describe remote replication support in each functional layer of CoprHD in more details.

UI/portal

In "Create Block Virtual Pool" dialog add "Remote Replication" data protection option. Enable user to specify target virtual array and optionally target virtual pool for remote replication data protection. Use this data as RemoteReplicationProtectionParam in virtual pool create request.

In "Create Block Volume" dialog add "Remote Replication Parameters" section where user may specify remote replication group for volumes. UI will use virtual array and virtual pool to identify the matching remote replication set for volume provisioning.  If the remote replication set supports only groups, user must specify existing group. If remote replication set supports groups and pairs, group selection is optional — if not specified, a new pair of volumes will be created directly in the remote replication set, outside of group container. Allow user to specify remote replication mode for volume pairs when created outside of group container.

Add new catalog directory entry for "Remote Replication Management" for remote replication management operations.Operations: establish, suspend, resume, split, swap, failover, failback, and move pairs from one replication groups to another replication group.


Api Service

Support in api service provides the following:

  • new RemoteReplicationSerive to support REST API to cover:  
    • discovery of  remote replication sets for specified storage type;
    • REST api for remote replication link  management operations;
    • REST api to create remote replication group
  • New RemoteReplicationProtectionParam for virtual pool create REST API ; 
    • This parameter specifies target virtual array, target virtual pool in virtual pools which require RemoteReplication protection. 
  • New attribute matcher class to match storage pools which can be used for remotely replicated source volumes in virtual pool:
    • The matcher class should allow only pools which a) belong to a source system in one of  remote replication sets; and  b) the remote replication set, which contain the source system, has target system with storage in target varray/vpool.
  • New RemoteReplicationParam in  "volumeCreate" REST api;
    • This parameter specifies existing RRG for volume pairs. When RRG is specified, volume pair will be placed in this replication group, otherwise, volume pair will be placed directly to RRS (matched by virtual array and virtual pool specified in volume create request) if this is supported by the set. 
    • RRG should be specified in case if driver supports only remote replication groups as volume containers, otherwise, when driver supports both groups and pairs, RRG can be optionally specified when volumes have to be placed in a group.
    • Remote replication mode
  • New RemoteReplicationSheduler to schedule and place remotely replicated volumes; 
    • Remotely  replicated volumes are always created in pre-configured RRS or in RRG (which has one source and one target system by default) as is specified in RemoteReplicationParam in volume create request. This means that source and target storage systems are already known from the request. Once storage systems for placement are known, volumes are placed in candidate storage pools in this systems the same way as it is done for regular volumes.
  • New RemoteReplicationBlockServiceApiImpl extension class of AbstractBlockServiceApiImpl to process placement recommendations and build volume descriptors for remotely replicated volumes;


Remote Replication Service REST API

Class/ REST Method /user roleDescriptionRequest parameter

RemoteReplicationGroupService/

POST

/vdc/block/remotereplicationgroups/create-group


This is system admin operation.

Create empty replication group. Request body specifies properties of the replication group.

 {

    "display_name" :  "Group01",

    "replication_mode" : "synchronous",

     "replication_state" : "active",

     "enforce_group_consistency" : "true",

     "source_system" : "storageos:storagesystem:1234-5678",

     "target_system" : "storageos:storagesystem:1234-5679"

 "source_ports":["portId_1", "portId_2"],

"target_ports":["portId_1", "portId_2"],

}


   

    

Remote Replication Link Management Operations.

All link management operations are System Admin operations.

  

RemoteReplicationManagementService/

POST /vdc/block/remotereplication/discover/{storage_type_uri}


This is system admin operation.

Discovers remote replication configuration for storage system type specified by 'type' uri

 {

    "context" :  operation context:

RR_SET,
RR_GROUP,
RR_PAIR,
RR_GROUP_CG,
RR_SET_CG

    "ids":  list of replication pairs ids,

}

RemoteReplicationManagementService/

POST /vdc/block/remotereplicationmanagement/establish

Establish (start) remote replication link

The same as above.

RemoteReplicationManagementService/

 POST /vdc/block/remotereplicationmanagement/suspend

 Suspend remote replication linkThe same as above.

RemoteReplicationManagementService/ 

POST /vdc/block/remotereplicationmanagement/split

 Split remote replication linkThe same as above.

 RemoteReplicationManagementService/

POST /vdc/block/remotereplicationmanagement/resume

 Resume remote replication linkThe same as above.

RemoteReplicationManagementService/

POST /vdc/block/remotereplicationmanagement/failover

 Failover remote replication linkThe same as above.

RemoteReplicationManagementService/

 POST /vdc/block/remotereplicationmanagement/failback

 Failback remote replication linkThe same as above.
 POST /vdc/block/remotereplicationmanagement/swap Swap remote replication linkThe same as above.


RemoteReplicationParameters for volume create request:

Create Pairs inside Replication GroupCreate Pairs outside of replication group

 {

    "replication_group": "storageos:remotereplicationgroup:1234-5678",

     "replication_mode" : "synchronous",

     "create_active" : "true"

}

 {

    "replication_mode" : "synchronous",

     "create_active" : "true"

}


Cassandra Data Model

Defines persistent types for remote replication sets, groups, pairs and elements with indexing for fast retrieval of child elements.

Stores remote replication protection related data in virtual pool instance.

Model classes:

  • RemoteReplicationSet
RemoteReplicationSet.java
@Cf("RemoteReplicationSet")
public class RemoteReplicationSet extends DiscoveredDataObject {

public enum ElementType {
REPLICATION_SET,
REPLICATION_GROUP,
REPLICATION_PAIR

}

public enum ReplicationState {
ACTIVE,
SYNCHRONIZING,
SUSPENDED,
SPLIT,
FAILED_OVER,
SWAPPED

}

// native id of replication set.

private String nativeId;

// Device label of this replication set

private String deviceLabel;

// If replication set is reachable.

private Boolean reachable;

// Type of storage systems in this replication set.

private String storageSystemType;

// Map of nativeId of storage system to its roles in the replication set.

private StringSetMap systemToRolesMap;

// Element types in this remote replication set for which which device supports replication link operations.

private StringSet supportedReplicationLinkGranularity;

// Set of replication modes which are supported for elements of this set

private StringSet supportedReplicationModes;

// Set of replication modes for elements of this set for which group consistency for
 // link operations is automatically enforced by device.

private StringSet replicationModesGroupConsistencyEnforced;

// Set of replication modes for which group consistency cannot be enforced on device.

private StringSet replicationModesNoGroupConsistency;

// When replication link operations are supported on the SET level, defines link mode.

private String replicationMode;

// When replication link operations are supported on the SET level, defines state of the link for this set.

private ReplicationState replicationState;

// Element types supported by this replication set.

private StringSet supportedElementTypes;


@Name("nativeId")
public String getNativeId() {
return nativeId;
    }

public void setNativeId(String nativeId) {
this.nativeId = nativeId;
        setChanged("nativeId");
    }

@Name("reachable")
public Boolean getReachable() {
return reachable == null ? false : reachable;
    }

public void setReachable(final Boolean reachable) {
this.reachable = reachable;
        setChanged("reachable");
    }

@AlternateId("AltIdIndex")
@Name("storageSystemType")
public String getStorageSystemType() {
return storageSystemType;
    }


public void setStorageSystemType(String storageSystemType) {
this.storageSystemType = storageSystemType;
        setChanged("storageSystemType");
    }

@Name("deviceLabel")
public String getDeviceLabel() {
return deviceLabel;
    }

public void setDeviceLabel(String deviceLabel) {
this.deviceLabel = deviceLabel;
        setChanged("deviceLabel");
    }

@Name("systemToRolesMap")
public StringSetMap getSystemToRolesMap() {
return systemToRolesMap;
    }

public void setSystemToRolesMap(StringSetMap systemToRolesMap) {
this.systemToRolesMap = systemToRolesMap;
        setChanged("systemToRolesMap");
    }

public void addSystemRolesEntry(String systemName, StringSet roles) {
if (this.systemToRolesMap == null) {
this.systemToRolesMap = new StringSetMap();
        }
this.systemToRolesMap.put(systemName, roles);
    }

@Name("supportedReplicationLinkGranularity")
public StringSet getSupportedReplicationLinkGranularity() {
return supportedReplicationLinkGranularity;
    }

public void setSupportedReplicationLinkGranularity(StringSet supportedReplicationLinkGranularity) {
this.supportedReplicationLinkGranularity = supportedReplicationLinkGranularity;
        setChanged("supportedReplicationLinkGranularity");
    }

@Name("supportedReplicationModes")
public StringSet getSupportedReplicationModes() {
return supportedReplicationModes;
    }

public void setSupportedReplicationModes(StringSet supportedReplicationModes) {
this.supportedReplicationModes = supportedReplicationModes;
        setChanged("supportedReplicationModes");
    }

@Name("replicationModesGroupConsistencyEnforced")
public StringSet getReplicationModesGroupConsistencyEnforced() {
return replicationModesGroupConsistencyEnforced;
    }

public void setReplicationModesGroupConsistencyEnforced(StringSet replicationModesGroupConsistencyEnforced) {
this.replicationModesGroupConsistencyEnforced = replicationModesGroupConsistencyEnforced;
        setChanged("replicationModesGroupConsistencyEnforced");
    }


@Name("replicationModesNoGroupConsistency")
public StringSet getReplicationModesNoGroupConsistency() {
return replicationModesNoGroupConsistency;
    }

public void setReplicationModesNoGroupConsistency(StringSet replicationModesNoGroupConsistency) {
this.replicationModesNoGroupConsistency = replicationModesNoGroupConsistency;
        setChanged("replicationModesNoGroupConsistency");
    }

@Name("replicationMode")
public String getReplicationMode() {
return replicationMode;
    }

public void setReplicationMode(String replicationMode) {
this.replicationMode = replicationMode;
        setChanged("replicationMode");
    }

@Name("replicationState")
public ReplicationState getReplicationState() {
return replicationState;
    }

public void setReplicationState(ReplicationState replicationState) {
this.replicationState = replicationState;
        setChanged("replicationState");
    }

@Name("supportedElementTypes")
public StringSet getSupportedElementTypes() {
return supportedElementTypes;
    }

public void setSupportedElementTypes(StringSet supportedElementTypes) {
this.supportedElementTypes = supportedElementTypes;
        setChanged("supportedElementTypes");
    }
}

RemoteReplicationGroup 

RemoteReplicationGroup.java
@Cf("RemoteReplicationGroup")
public class RemoteReplicationGroup extends DiscoveredDataObject {

    // native id of this group

private String nativeId;

    // If replication group is reachable.

private Boolean reachable;

    // Device label of this replication group.

private String deviceLabel;

    // Type of storage systems in this replication group.

private String storageSystemType;

    // Display name of this replication group (whem provisioned by the systemt).

private String displayName;

    // Source storage system of this group

private URI sourceSystem;

    // Target storage system of this group

private URI targetSystem;

    // replication mode of this group

private String replicationMode;

    // Element types for this replication group for which device supports replication link operations.
 // Can be group only, pair only or both types.

private StringSet supportedReplicationLinkGranularity;

    // replication state of this group

private RemoteReplicationSet.ReplicationState replicationState;

    // Defines if group consistency of link operations is enforced.

private Boolean isGroupConsistencyEnforced;

    // Parent replication set

URI replicationSet;

    @Name("nativeId")
    public String getNativeId() {
        return nativeId;
    }

    public void setNativeId(String nativeId) {
        this.nativeId = nativeId;
        setChanged("nativeId");
    }

    @Name("reachable")
    public Boolean getReachable() {
        return reachable == null ? false : reachable;
    }

    public void setReachable(final Boolean reachable) {
        this.reachable = reachable;
        setChanged("reachable");
    }

    @AlternateId("AltIdIndex")
    @Name("storageSystemType")
    public String getStorageSystemType() {
        return storageSystemType;
    }


    public void setStorageSystemType(String storageSystemType) {
        this.storageSystemType = storageSystemType;
        setChanged("storageSystemType");
    }

    @Name("deviceLabel")
    public String getDeviceLabel() {
        return deviceLabel;
    }

    public void setDeviceLabel(String deviceLabel) {
        this.deviceLabel = deviceLabel;
        setChanged("deviceLabel");
    }


    @Name("displayName")
    public String getDisplayName() {
        return displayName;
    }

    public void setDisplayName(String displayName) {
        this.displayName = displayName;
        setChanged("displayName");
    }

    @Name("sourceSystem")
    public URI getSourceSystem() {
        return sourceSystem;
    }

    public void setSourceSystem(URI sourceSystem) {
        this.sourceSystem = sourceSystem;
        setChanged("sourceSystem");
    }

    @Name("targetSystem")
    public URI getTargetSystem() {
        return targetSystem;
    }

    public void setTargetSystem(URI targetSystem) {
        this.targetSystem = targetSystem;
        setChanged("targetSystem");
    }

    @Name("isGroupConsistencyEnforced")
    public Boolean getIsGroupConsistencyEnforced() {
        return isGroupConsistencyEnforced;
    }

    public void setIsGroupConsistencyEnforced(Boolean isGroupConsistencyEnforced) {
        this.isGroupConsistencyEnforced = isGroupConsistencyEnforced;
        setChanged("isGroupConsistencyEnforced");
    }

    @Name("replicationMode")
    public String getReplicationMode() {
        return replicationMode;
    }

    public void setReplicationMode(String replicationMode) {
        this.replicationMode = replicationMode;
        setChanged("replicationMode");
    }

    @Name("replicationState")
    public RemoteReplicationSet.ReplicationState getReplicationState() {
        return replicationState;
    }

    public void setReplicationState(RemoteReplicationSet.ReplicationState replicationState) {
        this.replicationState = replicationState;
        setChanged("replicationState");
    }

    @RelationIndex(cf = "RelationIndex", type = RemoteReplicationSet.class)
    @Name("replicationSet")
    public URI getReplicationSet() {
        return replicationSet;
    }

    public void setReplicationSet(URI replicationSet) {
        this.replicationSet = replicationSet;
        setChanged("replicationSet");
    }

    @Name("supportedReplicationLinkGranularity")
    public StringSet getSupportedReplicationLinkGranularity() {
        return supportedReplicationLinkGranularity;
    }

    public void setSupportedReplicationLinkGranularity(StringSet supportedReplicationLinkGranularity) {
        this.supportedReplicationLinkGranularity = supportedReplicationLinkGranularity;
        setChanged("supportedReplicationLinkGranularity");
    }
}
  • RemoteReplicationPair
RemoteReplicationPair.java
@Cf("RemoteReplicationPair")
public class RemoteReplicationPair extends DataObject {

    public enum ElementType {
        VOLUME,
        FILE_SYSTEM

}

    // Element type (block or file element)

private ElementType elementType;

    // Device nativeId of replication pair.

private String nativeId;

    // If replication pair is part of replication group should be set to replication group URI, otherwise null.

private URI replicationGroup;

    // Either direct replication set parent or replication set of the replication group parent.

private URI replicationSet;

    // Replication mode of this pair.

private String replicationMode;

    // Replication state of this pair.

private RemoteReplicationSet.ReplicationState replicationState;

    // Replication pair source element.

private URI sourceElement;

    // Replication pair target element.

private URI targetElement;

    @Name("nativeId")
    public String getNativeId() {
        return nativeId;
    }

    public void setNativeId(String nativeId) {
        this.nativeId = nativeId;
        setChanged("nativeId");
    }

    @RelationIndex(cf = "RelationIndex", type = RemoteReplicationGroup.class)
    @Name("replicationGroup")
    public URI getReplicationGroup() {
        return replicationGroup;
    }

    public void setReplicationGroup(URI replicationGroup) {
        this.replicationGroup = replicationGroup;
        setChanged("replicationGroup");
    }

    @RelationIndex(cf = "RelationIndex", type = RemoteReplicationSet.class)
    @Name("replicationSet")
    public URI getReplicationSet() {
        return replicationSet;
    }

    public void setReplicationSet(URI replicationSet) {
        this.replicationSet = replicationSet;
        setChanged("replicationSet");
    }

    @Name("replicationMode")
    public String getReplicationMode() {
        return replicationMode;
    }

    public void setReplicationMode(String replicationMode) {
        this.replicationMode = replicationMode;
        setChanged("replicationMode");
    }

    @Name("replicationState")
    public RemoteReplicationSet.ReplicationState getReplicationState() {
        return replicationState;
    }

    public void setReplicationState(RemoteReplicationSet.ReplicationState replicationState) {
        this.replicationState = replicationState;
        setChanged("replicationState");
    }

    @RelationIndex(cf = "SourceElementOfReplicationPairIndex", type = DataObject.class, types = {Volume.class, FileShare.class})
    @Name("sourceElement")
    public URI getSourceElement() {
        return sourceElement;
    }

    public void setSourceElement(URI sourceElement) {
        this.sourceElement = sourceElement;
        setChanged("sourceElement");
    }

    @RelationIndex(cf = "TargetElementOfReplicationPairIndex", type = DataObject.class, types = {Volume.class, FileShare.class})
    @Name("targetElement")
    public URI getTargetElement() {
        return targetElement;
    }

    public void setTargetElement(URI targetElement) {
        this.targetElement = targetElement;
        setChanged("targetElement");
    }

    @Name("elementType")
    public ElementType getElementType() {
        return elementType;
    }

    public void setElementType(ElementType elementType) {
        this.elementType = elementType;
        setChanged("elementType");
    }
}


  • RemoteReplicationElement
RemoteReplicationElement.java
@Cf("RemoteReplicationElement")
public class RemoteReplicationElement extends DataObject {

    public enum ElementType {
        VOLUME,
        FILE_SYSTEM

}

    // uri of backing data object

private URI dataObjectUri;

    // Element type

private ElementType elementType;


    @Name("dataObjectUri")
    public URI getDataObjectUri() {
        return dataObjectUri;
    }

    public void setDataObjectUri(URI dataObjectUri) {
        this.dataObjectUri = dataObjectUri;
        setChanged("dataObjectUri");
    }

    @Name("elementType")
    public ElementType getElementType() {
        return elementType;
    }

    public void setElementType(ElementType elementType) {
        this.elementType = elementType;
        setChanged("elementType");
    }
}


Device Controller

Processes dispatcher requests to provision remote replication groups and pairs, processes remote replication link management requests.

Interfaces:

  • RemoteReplicationController.java
RemoteReplicationController.java
public interface RemoteReplicationController extends Controller {

    /**
 * Create empty remote replication group.
 * @param replicationGroup URI of remote replication group to create.
 */

public void createRemoteReplicationGroup(URI replicationGroup, String opId);

    /**
 * Create replication pairs in existing replication group container.
 * At the completion of this operation all remote replication pairs should be associated to their group and should
 * be in the following state:
 * a) createActive is true:
 * Pairs state: ACTIVE;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2;
 * replication links are set to ready state;
 *
 * b) createActive is false:
 * Pairs state: SPLIT;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are set to not ready state;
 * The state of replication pairs is same as after 'split' operation.
 *
 * @param replicationPairs list of replication pairs URI to create
 * @param createActive true, if pairs should start replication link automatically after creation (link in ready state), false otherwise
 *
 */

public void createGroupReplicationPairs(List<URI> replicationPairs, boolean createActive, String opId);

    /**
 * Create replication pairs in existing replication set. Pairs are created outside of group container.
 * At the completion of this operation all remote replication pairs should be associated to their set and should
 * be in the following state:
 * a) createActive is true:
 * Pairs state: ACTIVE;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2;
 * replication links are set to ready state;
 *
 * b) createActive is false:
 * Pairs state: SPLIT;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are set to not ready state;
 * The state of replication pairs is same as after 'split' operation.
 *
 * @param replicationPairs list of replication pairs to create
 * @param createActive true, if pairs should start replication link automatically after creation (link in ready state), false otherwise
 */

public void createSetReplicationPairs(List<URI> replicationPairs, boolean createActive, String opId);

    /**
 * Delete remote replication pairs. Should not delete backend volumes.
 * Only should affect remote replication configuration on array.
 * At the completion of this operation all remote replication elements from the pairs
 * should be in the following state:
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are not ready;
 * R1 and R2 elements are disassociated from their remote replication containers and
 * become independent storage elements;
 *
 * @param replicationPairs replication pairs to delete
 */

public void deleteReplicationPairs(List<URI> replicationPairs, String opId);


    // replication link operations
 /**
 * Suspend remote replication link for remote replication argument.
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: SUSPENDED;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * replication links should be in not ready state;
 *
 * @param replicationArgument: set/group/pair
 */

public void suspend(URI replicationArgument, String opId);

    /**
 * Resume remote replication link for remote replication argument.
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2 elements;
 * replication links should be in ready state;
 *
 * @param replicationArgument: set/group/pair
 */

public void resume(URI replicationArgument, String opId);

    /**
 * Split remote replication link for remote replication argument.
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: SPLIT;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links should be in not ready state;
 *
 * @param replicationArgument: set/group/pair
 */

public void split(URI replicationArgument, String opId);

    /**
 * Establish replication link for remote replication argument.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data;
 * replication links should be in ready state;
 *
 * @param replicationArgument replication argument: set/group/pair
 */

public void establish(URI replicationArgument, String opId);

    /**
 * Failover remote replication link for remote replication argument.
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: FAILED_OVER;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be write disabled;
 * R2 elements should be read/write enabled;
 * replication links should be in not ready state;
 *
 * @param replicationArgument: set/group/pair
 */

public void failover(URI replicationArgument, String opId);

    /**
 * Failback remote replication link for remote replication argument.
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state:ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be write disabled;
 * data on R1 elements is synchronized with new R2 data copied to R1;
 * replication links should be in ready state;
 *
 * @param replicationArgument: set/group/pair
 */

public void failback(URI replicationArgument, String opId);

    /**
 * Swap remote replication link for remote replication argument.
 * Changes roles of replication elements in each replication pair.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: SWAPPED;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read enabled/write disabled;
 * R2 elements should be read/write enabled;
 * data on R2 elements is synchronized with new R1 data copied to R2;
 * replication links should be in ready state for replication from R2 to R1;
 *
 * @param replicationArgument: set/group/pair
 */

public void swap(URI replicationArgument, String opId);

    /**
 * Move replication pair from its parent group to other replication group.
 * At the completion of this operation remote replication pair should be in the same state as it was before the
 * operation. The only change should be that the pair changed its parent replication group.
 *
 * @param replicationPair replication pair to move
 * @param targetGroup new parent replication group for the pair
 */

public void movePair(URI replicationPair, URI targetGroup, String opId);
}

Classes:

RemoteReplicationDeviceController: implements workflow steps to provision remote replication links for volume pairs (implements BlockOrchestrationInterface); processes remote replication link management requests.

With modular architecture supported by SB SDK  most of platform specific features of remote replication operations are done by drivers. Based on this architecture device controller will be a thin layer. For example, all steps required to delete replication pair (pause, detach, stop, remove from group, etc.)  will be done by driver, not by controller.  

BlockOrchestrationDeviceController: calls RemoteReplicationDeviceController to add remote replication related steps to volume provisioning workflow.


External Storage Device

Implements device level support for remote replication operations. Serves as an adaptor between CoprHD controller and storage drivers.

Interfaces:

  • RemoteReplicationDevice.lava
RemoteReplicationDevice.java
public interface RemoteReplicationDevice {

    public void createGroupReplicationPairs(List<RemoteReplicationPair> replicationPairs, TaskCompleter taskCompleter);
    public void createSetReplicationPairs(List<RemoteReplicationPair> replicationPairs, TaskCompleter taskCompleter);
    public void deleteReplicationPair(URI replicationPair, TaskCompleter taskCompleter);

    // replication link operations

public void start(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void stop(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void suspend(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void resume(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void split(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void synchronize(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void failover(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void failback(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);
    public void swap(RemoteReplicationArgument replicationArgument, TaskCompleter taskCompleter);

}

Classes:

  • ExternalDeviceCommunicationInterface: discovers remote replication configuration for providers/arrays managed by drivers. Configuration elements are remote replication sets and remote replication groups.
  • ExternalBlockStorageDevice: implements methods for provisioning of remote replication pairs; implements methods for management of remote replication links between three types of remote replication elements: pairs, groups, sets.


Storage Driver SDK

Support in SDK includes the following:

  • Set of remoter replication related model classes:
    • RemoteReplicationSet
    • RemoteReplicationGroup 
    • RemoteReplicationPair
    • RemoteReplicationMode
  • Discovery API for discovery of remote replication sets and remote replication groups
  • API for provisioning of remote replication groups and remote replication pairs
  • API to manage links between remote replication pairs: establish, suspend,  resume, split, resume, failover, failback, swap
  • API to delete remote replication pairs: splits links and make all source and target devices independent read/write devices outside of any remote replication containers  

Interfaces:

  • RemoteReplicaitonDriver.java
  • DiscoveryDrirver.java
DiscoveryDriver.java
/** * Discover remote replication sets managed by driver. * * @param storageSystemNativeIds storage systems managed by driver. Type: Input. * @param storageProviderNativeIds storage providers managed by driver. Type: Input. * @param remoteReplicationSets remote replication sets managed by driver. Type: Output. * @return driver task */ public DriverTask discoverRemoteReplicationSets(List<String> storageSystemNativeIds, List<String> storageProviderNativeIds, List<RemoteReplicationSet> remoteReplicationSets);
RemoteReplicationDriver.java
public interface RemoteReplicationDriver {

    /**
 * Create empty remote replication group.
 * @param replicationGroup specifies properties of remote replication group to create.
 * @param capabilities storage capabilities for the group
 * @return driver task
 */

public DriverTask createRemoteReplicationGroup(RemoteReplicationGroup replicationGroup, StorageCapabilities capabilities);

    /**
 * Create replication pairs in existing replication group container.
 * At the completion of this operation all remote replication pairs should be associated to their group and should
 * be in the following state as defined in the replicationPair argument:
 * a) Pairs state: ACTIVE;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2;
 * replication links are set to ready state;
 *
 * b) Pairs state: SPLIT;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are set to not ready state;
 * The state of replication pairs is same as after 'split' operation.
 *
 * @param replicationPairs list of replication pairs to create; each pair is in either active or split state
 * @param capabilities storage capabilities for the pairs
 * @return driver task
 */

public DriverTask createGroupReplicationPairs(List<RemoteReplicationPair> replicationPairs, StorageCapabilities capabilities);

    /**
 * Create replication pairs in existing replication set. Pairs are created outside of group container.
 * At the completion of this operation all remote replication pairs should be associated to their set and should
 * be in the following state:
 * a) Pairs state: ACTIVE;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2;
 * replication links are set to ready state;
 *
 * b) Pairs state: SPLIT;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are set to not ready state;
 * The state of replication pairs is same as after 'split' operation.
 *
 * @param replicationPairs list of replication pairs to create; each pair is in either active or split state
 * @param capabilities storage capabilities for the pairs
 * @return driver task
 */

public DriverTask createSetReplicationPairs(List<RemoteReplicationPair> replicationPairs, StorageCapabilities capabilities);

    /**
 * Delete remote replication pairs. Should not delete backend volumes.
 * Only should affect remote replication configuration on array.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication elements from the pairs
 * should be in the following state:
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links are not ready;
 * R1 and R2 elements are disassociated from their remote replication containers and
 * become independent storage elements;
 *
 * @param replicationPairs replication pairs to delete
 * @return driver task
 */

public DriverTask deleteReplicationPairs(List<RemoteReplicationPair> replicationPairs);


    // replication link operations

 /**
 * Suspend remote replication link for remote replication pairs.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs should be in the following state:
 * Pairs state: SUSPENDED;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * replication links should be in not ready state;
 *
 * @param replicationPairs: remote replication pairs to suspend
 * @return driver task
 */

public DriverTask suspend(List<RemoteReplicationPair> replicationPairs);

    /**
 * Resume remote replication link for remote replication argument.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data copied to R2 elements;
 * replication links should be in ready state;
 *
 * @param replicationPairs: remote replication pairs to resume
 * @return driver task
 */

public DriverTask resume(List<RemoteReplicationPair> replicationPairs);

    /**
 * Split remote replication link for remote replication pairs.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: SPLIT;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read/write enabled;
 * replication links should be in not ready state;
 *
 * @param replicationPairs: remote replication pairs to split
 * @return driver task
 */

public DriverTask split(List<RemoteReplicationPair> replicationPairs);

    /**
 * Establish replication link for remote replication argument.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be read enabled/write disabled;
 * data on R2 elements is synchronized with R1 data;
 * replication links should be in ready state;
 *
 * @param replicationPairs remote replication pairs to establish
 * @return driver task
 */

public DriverTask establish(List<RemoteReplicationPair> replicationPairs);

    /**
 * Failover remote replication link for remote replication argument.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: FAILED_OVER;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be write disabled;
 * R2 elements should be read/write enabled;
 * replication links should be in not ready state;
 *
 * @param replicationPairs: remote replication pairs to failover
 * @return driver task
 */

public DriverTask failover(List<RemoteReplicationPair> replicationPairs);

    /**
 * Failback remote replication link for remote replication argument.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state:ACTIVE;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read/write enabled;
 * R2 elements should be write disabled;
 * data on R1 elements is synchronized with new R2 data copied to R1;
 * replication links should be in ready state;
 *
 * @param replicationPairs: remote replication pairs to failback
 * @return driver task
 */

public DriverTask failback(List<RemoteReplicationPair> replicationPairs);

    /**
 * Swap remote replication link for remote replication argument.
 * Changes roles of replication elements in each replication pair.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation all remote replication pairs which belong to the replication argument should
 * be in the following state:
 * Pairs state: SWAPPED;
 * No change in remote replication container (group/set) for the pairs;
 * R1 elements should be read enabled/write disabled;
 * R2 elements should be read/write enabled;
 * data on R2 elements is synchronized with new R1 data copied to R2;
 * replication links should be in ready state for replication from R2 to R1;
 *
 * @param replicationPairs: remote replication pairs to swap
 * @return driver task
 */

public DriverTask swap(List<RemoteReplicationPair> replicationPairs);

    /**
 * Move replication pair from its parent group to other replication group.
 * Should not make any impact on replication state of any other existing replication pairs which are not specified
 * in the request. If execution of the request with this constraint is not possible, should return a failure.
 *
 * At the completion of this operation remote replication pair should be in the same state as it was before the
 * operation. The only change should be that the pair changed its parent replication group.
 *
 * @param replicationPair replication pair to move
 * @param targetGroup new parent replication group for the pair
 * @return driver task
 */

public DriverTask movePair(RemoteReplicationPair replicationPair, RemoteReplicationGroup targetGroup);
}

Functional Requirements

Support provisioning and management (link operations) of remotely protected elements on storage arrays managed by SB SDK drivers.


JIRA ID1-Line DescriptionComments, notes, etc.
 

COP-23450

 SB SDK: Implement support for discovery of RemoteReplicationSets On demand and periodic discovery of remote replication sets and groups for driver managed storage array types available in the system. Should collect enough data about properties of remote replication sets and groups to enable system to learn how to provision remotely replicated pairs of volumes on the arrays of the given type. 
 

COP-23452

SB SDK: Implement apisvc layer to provision remotely replicated volumes  This includes required changes in virtual pool to support remote replication protection (RemoteReplicationProtectionParam), addition of RemoteReplicationParam to volume create request, implementation of RemoteReplicationServiceApiImpl, new pool matcher to filter out storage pools outside of remote replication sets and remote replication groups(if group is specified) in varrays. Could be divided to subtasks.
 

COP-23451

 SB SDK: Implement RemoteReplicationSchedulerPlacement implementation of remotely replicated volumes according to remote replication set and remote replication group in volume create request. 
 

COP-23560

 SB SDK: UI/Portal support for provisioning remotely replicated volumes Support new data protection option in virtual pool create dialog; support new remote replication parameter for volume create dialog; new catalog entry for remote replication link operations. 
 

COP-23455

 SB SDK: Implement controller layer for Remote Replication Provisioning 

BlockOrchestrationDeviceController: add support to call RemoteReplicationDeviceController when volume require remote replication protection.

New RemoteReplicationDeviceController:  implement create remote replication link step for remote replication pairs for volume (implements BlockOrchestrationInterface), process remote replication link management requests;

 

COP-23457

 SB SDK: Implement device level support to provision remotely replicated volumes Implements device level support for remote replication operations. Translates controller calls to device driver calls. Serves as an adaptor between CoprHD controller and storage drivers.
 

COP-23235

 SB SDK: Define interface and class structure for remote replication Sceleton for Remote replication support: Define SB SDK model classes and api methods. Define device level apis and classes. Define data base model for remote replication. Define controller interface and classes. Define apisvc components — apisvc methods, placement scheduler, RemoteReplicationServiceApi.
COP-26138  SB SDK: Implement Cli for for remote replication supportImplement Cli support for remote replication protection and for remote replication link management operations. 
COP-26138  SB SDK: Implement Cli for for remote replication supportImplement Cli support for remote replication protection and for remote replication link management operations. 

Exclusions and Limitations

  1. Configuration of RRSs is considered to be part of configuration planning for remote replication in physical domain and should be done outside of CoprHD. CoprHD provisions volumes with already existing RRSs.
  2. We plan to support only RRS with 1_source-to-1_target systems for phase 1 release. Support for other RRSs configurations is targeted for the future.  
  3. When user does not specify existing RRG for remotely protected volumes, we always create protected pair directly in RRS.
  4. The assumption is that remote replication containers for volumes on device are independent from volume containers used locally. This guarantees that there is no impact between operations on remote replication containers and operations on local containers. For example, when volumes are provisioned in remote replication groups, there is no group impact for local volume operations (snaps, clones, local mirrors). Local operations can be applied to volumes as defined by their local groups. Remote replication management REST API can be applied to remotely protected volumes in consistency groups.

Future Work and Related Projects

Support for virtual pool change request is TBD due to re-architecture planned for this functionality in Anakin and above. 

Ingestion of remotely replicated pairs of  storage volumes.

Support remotely replicated volumes in HP 3PAR, SVC and DELL storage drivers. 

Related project: Support for VMAX SRDF link management operations through remote replication link management API. Common API and UI Service Catalog for link management operations for third party arrays and VMAX SRDF. This project is under way and is scheduled for Leia.

Relate project: Support for RecoverPoint link management operations through remote replication link management API. Common API and UI Service Catalog for link management operations for third party arrays, VMAX SRDF and RP replicated volumes. Timeframe: TBD.


Implementation Strategy

Implementation Timeframe

Timeframe is Leia release.

Branch: https://review.coprhd.org/projects/CH/repos/coprhd-controller/compare/commits?sourceBranch=refs%2Fheads%2Ffeature-COP-23235-sdk-supportForRemoteReplication&targetBranch=refs%2Fheads%2Fmaster

Virtual Team

Development: 

  • All implementation tasks, except UI/portal: Evgeny  Roytman, Shiliang Cao
  • UI/portal: Steve Mendes
  • CLI: TBD
  • QE: Liu Peipei

Testing Strategy

This feature will be tested with the driver simulator. Driver simulator has to be extended to simulate remote replication elements.

Tests for remote replication implementation in storage drivers will also test SDK support for remoter replication.

Documentation Strategy

All new features for remote replication in UI and REST api have to be documented. 

Impact Evaluation

Public APIs and public functionality (end-user impact)

No impact to existing public apis and existing public functionality for natively managed storage systems. All  changes for this feature are only affect driver managed arrays. 

Upgrades

Do not immediately see upgrade impact. TBD.


Reviewed and Approved By

NameDateVoteComments, remarks, etc.
   
2017Apr24+1I had many rounds of discussion with Evgeny regarding the above proposal.All my concerns were taken into consideration and few had been implemented.Approving .
2017Dec06+1The comments section answered many of my questions. Remaining questions regarding flexibility of operations are annotated, but it overall looks great. Good job, as usual!






11/15/2016

04/30/2017

+0.5

+1

Initial reading. I have many questions, see comment from 11/15/2016. Will want to understand answers to questions before approving. Please uncheck me when answers are provided. Approved 4/30/2017.









  • The members of the CoprHD Technical Steering Committee (TSC) have been automatically added to your reviewer list
    • A majority of them must approve this document before the feature can be merged to a project integration branch (e.g. integration-*, release-*, master)
    • Please un-check the boxes next to their names when your document is ready for their review
  • Please refer to Design Approval Guidance for information on additional reviewers who should be added and the approval process











  • No labels

18 Comments

  1. Please add an appropriate QE reviewer and uncheck the reviewer names above when ready for review.

  2. Done. Ready for review.

  3. Evgeny, please conduct a meeting to review the design.  It will help start the review process.

  4. Initial reading review comments Nov. 15th, 2016.

    1. "Types of replication elements which can be provisioned in this set: remote replication pairs (dell, xiv, hds), remote replication groups (hp3par, xiv, hds) or both (dell, xiv, hds)". Seems wrong- if dell is both shouldn't it be in remote replication groups as well? This is a nit...
    2. A question: does a RRG map to a Vipr Consistency Group? Or is there no correlation?
    3. Question: "add new matcher class to AttributeMatcherFramework to support matching storage pools which can be used for remotely replicated elements;" Will this matcher be topology based? i.e. will it know from the RRS what two arrays can be connected, and use this in its matching algorithm?
    4. Question: "New RemoteReplicationSheduler to schedule and place remotely replicated volumes; " Given we will be supporting VMAX3 Rest with Southbound SDK, do you plan to implement scheduler so it can be used behind VPLEX for VPLEX+SRDF? Behind RP (maybe not needed)?
    5. More questions... "Local volume groups do not overlap with groups for the purpose of remote replication." I'm still trying to figure out what the user experience will look like. Do do not anywhere mention a link of Consistency Group to RRG. Will that exist? If not, if I go to GUI screens, when I choose something like failover where today I can select a volume or consistency group, will I instead see three potential types of choices, volume, CG, and RRG? If so I'm not sure this is the model we want. That is to say perhaps the CG virtualizes a physical RRG in my mind. I would like some discussion of this please.

    I'm inclined to approve, but I have many questions and want them answered so I can think about the implications please uncheck my task when answers are provided.

  5. These are my answers: at the same numbers as questions.

    1. Yes, fixed.
    2.  There is no correlation.
    3. Yes, this is topology based. The matcher class should allow only pools which a) belong to a source system in one of  the RRSs; and  b) the remote replication set, which contain the source system, has target system with storage in target varray/vpool. Since we plan to simplify virtual pools and extract topology information from it, I will put the implementation of this matcher on hold for the first release. Correct pools will be selected at volume provisioning time, based on the specified replication set and replication group.
    4. Yes, the scheduler should fit to the existing framework for chaining schedulers. I will add necessary configuration for remote replication scheduler, but probably not for the Anakin.
    5. There is no explicit relationship between RRGs and local consistency groups — they are managed independently. This is what I thought about user experience related to remote replication link operations:
      User selects link operation and storage type. Based on the supported granularities of link operations for the RRSs of this storage type, UI will display types of remote replication arguments for this storage type. UI user will be asked to select  RRS (set), RRG (group), or RRP (pair)  type, based on what is displayed. Based on the selected argument type, user will select specific element of this type for the operation. For example, assume user wants to failover RRG with device label Group_HR1 managed by driver of type XYZ (the same as storage type). When UI user selects "failover" as operation type and XYZ as storage type, we will check what are supported granularity of link operations for discovered RRSs of XYZ driver. List of supported argument types corresponding to the supported operation granularities will be displayed and user will check "RRG" type. UI will show all RRGs managed by XYZ driver which are valid for group failover operation (based on the group replication state). User will select the required one and execute the operation. You can think of RRG as CG and RRP as remotely protected volume in this context.  UI can be tailored to meet required user experience. I will be glad to discuss this.
    6.  
    1. Thanks, these are good answers to my questions, and I like your answer to #5 although it seems to complicate the GUI. We would need to abstract that somehow. Given we're nearing the end of the release, how close are you in the implementation of #5?

      1. We did not start implementation of UI support for remote replication link operations: failover, failback, and others. UI may not be ready for Anakin release. We may need to work with usability group to define UI design which will be good for user. API support and controller support for remote link operations is part of Anakin. API will provide REST interface to execute link operations for each of three remote replication element types: sets, groups, pairs.

        1. So given what you've said, is the UI support going to be there for Anakin that is described under the "UI Portal" section? Also will the matcher be available? I'm still a bit confused what will be provided in Anakin and what will come later. Thanks, Tom

  6. Evgeny, please reach out to reviewers to get approval.  Please add design review recording to wiki.

  7. Thank Bill, I  will. Design review meeting recording is in the attachment.

  8. TSC members need to review/approve/comment on this spec.  Please do so ASAP.

  9. Hi Evgeny,

    Some initial comments based on the spec reading. Haven't listened to the review recording – will do so soon.

    1. Remove any mention of file. Let's claim only block support right now. There are many more complications when doing file replication (directory levels, etc.) which need to be handled, before we can claim file support.
    2. Device Specific Data: If a tenant admin does provisioning operations, how is he supposed to know which device specific data to provide? This sounds not practical in the real deployment.
    3. APIs (URLs) should not start with /remotereplication. They can start with /block or /vdc.
    4. Remote replication Set is a physical resource (storage system pair, essentially). It should not be visible to tenant users. That breaks the CoprHD/ViPR abstraction model completely.
    5. For each API specify the role. It is not clear from the spec as to which API is meant for System Admin and which for Tenant users.
    6. We must not expose remote replication sets in 'create volume' type of catalog services. They must be encapsulated into vpools and varrays. Ultimately for volume creation we expect tenant users to provide only 5 parameters – name, size, vpool, varray and project. Adding any additional parameter is going to mean changes to all existing clients and their scripts. Think of how today customers can provision volumes on standalone arrays or VPLEX/RP/SRDF using the same REST API – just the vpool and varray differs. We need to have the same model when using remote replication via southbound SDK.
    7. In the absence of any real driver using the API, there is only so much review we can do. I expect one more revision of the API when a real driver starts using this feature post Anakin.

    (end of comments)

    Anil 

  10. Hi Anil,

    Thank you for your comments. These are my answers:

    1. Removed. SDK remote replication apis are generic, they do not mentioned physical element types. Implementation for block and file is different.
    2. This could be done as name-value pairs based on storage system type which is used for provisioning. The storage type is known either from virtual pool or from context of provisioning request.

    3. Agree. I implemented them as /vdc/block/remotereplication… . Need to update this spec with final api URIs .

    4. This needs discussion. For some array types remote replication elements are provisioned in remote replication set directly and we need a way to specify which set to use. Not all array types have concept of remote replication group. For such arrays this will require to have separate source and target virtual arrays for each remote replication set. 
    5. I will review this.
    6. I will need to think of this more. This is my reasoning. My concern was that to encapsulate details about remote replication sets, modes, groups in virtual arrays and virtual pools, means to further increase number of their instances and complicate these structures. We started to work on vpool/varray simplification and I moved such details to provisioning time, since there is no other way to keep this out of vpool/varray. One way or another we need to know which remote replication set, mode, group should be used to provision volumes. Different array types have different ways to implement remote replication and there is no standard model for all arrays. As an example, some array types do not have concept of remote replication group and remote replication modes are different for different array types. With srdf, groups are always required and they are provided indirectly through projects. We did not want to continue with this approach going forward, so I defined group explicitly at volume provisioning type. Since groups are not common for all arrays, I have a way to specify remote replication set as well.
    7. I agree with this. 


    Evgeny 

  11. Hi Evgeny:

        For create VMAX SRDF volume, we still need to use the "Remote Replication" data protection option in the virtual pool, the existing SRDF operation will not be considered, right?

  12. Currently there are no plans to remove existing SRDF protection option for VMAX SRDF  volumes. 

  13. Reread 4/30/2017. Is this targeted for Skywalker?

    I think you may want index for RRS and RRG deviceLabel field. Approving. Tom

    At what point will VMAX3 support be switched over to the new model? Only after driver is ported to SB-SDK? That is my assumption.

    I would like to see more information about testing strategy and test cases. Are the QE folks assigned to this project?

    Thanks, Tom


    1. >>> At what point will VMAX3 support be switched over to the new model? Only after driver is ported to SB-SDK? That is my assumption.<<<

      Yes. When we have VMAX3 SDK driver. For Leia we started to work on partial solution to integrate existing SRDF link management operations through common remote replication REST API and UI catalog service for link management. The reason for this is to have one REST API and one Catalog Service for SRDF link management and third part remote replication link management.

      >>> The project was rescheduled for Leia and Peipei will cover QE for SRDF integration and 3d party arrays. Peipei will prepare test plan. 

  14. Reviewed this document and make changes to align the document with how this feature is being implemented.

    Summary of changes:

    --- Remote replication set was removed from volume create UI dialog and volume create API. The remote replication set where a volume is placed is defined by virtual array and virtual pool in volume create request.

    --- Added user role definition to remote replication operations: volume create is tenant operation, remote replication group create and all remote replication link management operations are system administrator operations.

    --- Fixed REST API description.

    The document is ready for final review.