Skip to end of metadata
Go to start of metadata
 
  • Replace the gray cursive text with the actual content.
  • Remove non-applicable sections or insert N/A.

 


High Level Functional Overview and Design Specifications

Problem Statement

Release of CoprHD as open software product is expected to attract attention from 3-rd party storage vendors. It is anticipated that vendors will start integrating their arrays into CoprHD.

This requires to provide clear way to integrate support for 3-rd party storage array drivers into CoprHD.

In the past integration of new type of storage array to ViPR was done internally by ViPR engineering. Typically, new array types bring their own flavor of data services, unique capabilities, their own device export constructs. All these were primarily statically coded in ViPR. This required additions and modifications to several software components within ViPR core — UI, persistent classes, export orchestrators, placement matchers. There was no focus on clean separation of ViPR  core from device driver layer. Practically, layer of ViPR which talked to device APIs also performed ViPR database updates, managed ViPR task completion and executed other internal task communicating directly to ViPR core . The focus was on  utilizing device APIs to provide rich set of functionality to clients. Isolation of device access to its own layer and avoiding static coding of device specific dependencies into product core was not a priority. In this model, addition of new arrays to CoprHD supported set of storage products is challenging for open source community.

This is high-level description of limitations of current device access layer in CorpHD for open source storage array integration.

CoprHD has software layer which encapsulates access to physical devices through their APIs. This layer provides internal interfaces to CoprHD core for discovery, provisioning, metering and monitoring.

Interface Definition:

Declaration of device layer interfaces uses CoprHD persistent classes, primary keys of persistent classes and requires task completer instance with specific implementation to handle completion of the request. This implies that the device layer has direct access to CoprHD database.  Typical task completer in the call to device layer should update CoprHD database with physical information about provisioned resource and notify core about request completion. When operation is part of multistep workflow, task completers call workflow completer, which in its turn sends request to core workflow service to complete workflow step and to initiate other workflow steps which have been blocked on the completed step. Task completers are operation specific. From this description it is clear that declarations of device access layer in CoprHD have direct dependency on CoprHD infrastructure.

Interface Implementation:

Implementation of device layer API is tightly bound to Cassandra and Zookeeper, and in some cases needs access to instances of core CoprHD services for internal communication. Typically, implementation of these interfaces uses parameters, which reference persistent classes, to read necessary data from CoprHD database, builds request to physical device, executes this request and calls task completer to update database with physical properties of provisioned devices and to call CoprHD core services to notify about task completion.

The major issue with this approach is that interfaces and their implementation require knowledge of internal details about CoprHD persistent layer and CoprHD core services.

Device Discovery

Currently CoprHD does not have generic API and generic model to discover and to process data services and capabilities provided by storage devices in device independent way. Most of the capability information is statically coded in the persistent classes and across internal CoprHD services and CoprHD UI. The issue here is that unique capabilities of new storage arrays should be manually coded into CoprHD database, core services and UI.

Functional Requirements  

    • Define device driver SDK isolated from CoprHD persistent model and internal services. The SDK should not have any dependency on CoprHD infrastructure classes inside SDK interface declarations and SDK object model. The SDK should include clear APIs for CoprHD services available for device drivers.  In order for CoprHD to integrate the drivers, CoprHD should implement internal adapter to adapt device access layer to use new device API. Storage driver should implement interfaces according to the specification. All other processing related to CoprHD infrastructure will be done by the CoprHD adapter layer.

     

    • Implement generic model to define and to process capabilities of storage arrays managed by storage drivers.  Use this model as an input for CoprHD UI and CoprHD core components to support storage provisioning based on storage capabilities. The capability model has to supply all information needed for CoprHD to implement vPool/vArray driven placement and provisioning without static dependency on array types. Implement storage capability API.

     

JIRA ID1-Line DescriptionComments, notes, etc.
  Define storage driver SDKDefine SDK interfaces, object model, APIs for available CoprHD services. Provide jar with SDK specification. Provide driver developer's guide. Define performance requirements for storage drivers.
 

 Define standard service option(s) model in driver

SDK (CTD CoS/LoS model is a source)

Define class model for standard service options. Base this model on CTD proposal for standard CoS/LoS model. Add this model to driver API.
 COP-17123 Implement CoprHD support for storage discovery using storage driver Implement ExternalCommunicationInterface class to support integration of DiscoveryDriver API.
 COP-17123 Implement CoprHD support for block storage provisioning using storage driver Implement ExternalBlockStorageDevice class to support integration of BlockStorageDriver API.
 COP-17123 Implement support for ExternalCommunicationInterface and ExternalBlockStorageDevice classes Implement required code in controller service to support discovery and provisioning through ExternalCommunicationInterface and ExternalBlockStorageDevice classes.
 COP-17123 Implement CoprHD services provided to storage driversImplement LockMangerImpl, RegistryImpl classes.
  Implement support for service options (standard capabilities) Service options discovery, persistence, use of service options in provisioning stack
  Support runtime upload of storage driversImplement non-disruptive support for upload of storage drivers. Can be done from "System Management" service catalog.
  Capability model specificationDefine Java based description for capability definition and capability instance classes. Provide specification for integration of device driver capabilities in CoprHD storage provisioning stack (vPool definition, storage pool matching).
  Implement support for capability based storage provisioning This is implementation task to consume capabilities of storage drivers and use them in CoprHD storage provisioning.
  Portal support for provisioning with storage managed by drivers Implement UI support for provisioning storage to driver managed arrays
   

Design Approach / High Level Implementation Details

    • Hash-out the design and high level implementation ideas
    • Are there any alternative design one should consider? Is this the optimal one?
    • What are the user visible changes? Will this change break user's script or automation code? Will it impact CoprHD QE automation scripts?
    • What are the changes to CoprHD components and component interaction? Are teams responsible for these other components on boards with this change?
    • What are the changes to persistent data? Will it require special consideration during upgrades?
    • Please use consistent and precise nomenclature / terminology.
    • Please include diagrams and illustrations if applicable.

Exclusions and Limitations

    • Which requirements and/or functionality are deliberately excluded from this project?

Future Work and Related Projects

    • Is this change setting ground for a future project? In other words, are there projects that depend on tis change?
    • Does this change depend on other projects? 

Implementation Strategy

Implementation Timeframe

    • What are the milestones (code complete, code ready for review, code ready to commit)?
    • How much time will be needed to test this change and address all the issues after the initial commit?
    • Is this a multi-phase project?

Virtual Team

    • Who will work on this change?
    • Does the virtual team span multiple physical teams? If so, how are you going to share responsibilities and ensure consistency? 

Testing Strategy

    • How are you going to ensure good quality?
    • Does this change require any specific performance and scale testing?
    • Does this change require security scans?

Documentation Strategy

    • What are the necessary changes to CoprHD documentation?

Impact Evaluation

Public APIs and public functionality (end-user impact)

    • Is this change backward compatible? Could incompatibilities be avoided? If not, what changes will have customer and CoprHD QE apply to their automation software developed against the old APIs?

Other components and component interaction

    • If the change spans multiple components and/or changes interaction between components, can it create any new problems?
    • Does this change affect dependencies between services?
    • Does it change startup or shutdown order?
    • Does it require any new management interfaces?

Persistence model

 

    • Are schema changes need in dbsvc or coordinatorsvc?
    • If so, what is the upgrade procedure?Is the schema migration (conversion) going to be reliable and scalable?
    • Does this change affect disk usage?

Upgrades

    • Are there any other special considerations regarding upgrades? Think twice - this is rarely obvious (wink)

    • Will we need any new tools to debug or repair system if the upgrade code specific to this change fails?

Performance

    •  Can this change adversely affect performance? If so, how are you going to test it? Is there a way to test this early, before the entire implementation is ready?

Scalability and resource consumption

    • Will it scale? How long will essential operations take at the scale of O(10,000,000)? How are you going to test it?
    • Will specific performance at scale testing be required?
    • Does this change have impact on memory and CPU usage? How does memory and CPU usage scale with the number of objects?

Security

    • Are there any implications for CoprHD security?
    • Will new security scans be required?

Deployment and serviceability

 

Developer impact, dependencies and conditional inclusion

    • What is the impact on build time?
    • Is there any impact on integration with IDEs (Eclipse and IntelliJ)?
    • Are there any new third party dependencies? If so, please list these dependencies here:
Package NamePackage VersionLicenseURLDescription of the package and what it is used for
     
     
     
    • Shall this feature by included with a special build-time flag?
    • Shall this feature be enabled only under certain run-time condition?

Reviewed and Approved By

NameDateVoteComments, remarks, etc.
    
    
    
    





 

 

  • No labels