Automate Your BI istration to Save Millions with Command Manager and System Manager Presented by: Dennis Liao | Sr. Sales Engineer Date: 27th January, 2015
#mstrworld
Session 2 This Session is Part of
MicroStrategy World Track 6: Enterprise BI I
#mstrworld
MicroStrategy’s istration Product Suite MicroStrategy Leads with a Comprehensive Set of istration Tools
Source– Cindi Howson, BI Scorecard® Strategic and Product Summary
#mstrworld
MicroStrategy’s istration Product Suite MicroStrategy System Manager
Create Automated Workflows `
MicroStrategy Command Manager
Create Automated Scripts
MicroStrategy Enterprise Manager
Monitor Performance and Utilization
MicroStrategy Integrity Manager
Automate Regression Testing
MicroStrategy Object Manager
Coordinate Change Management
#mstrworld
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
The Need for Automated istrative Processes Manage Daily Operations Monitor Activity
Upgrade Metadata
Manage Security
Troubleshoot System Errors
Ensure Data Integrity
Fine-tune BI System
Modify Intelligence Server Settings
Manage Scheduled Requests
Bulk Manipulation of Objects
#mstrworld
THE PROBLEM: Most Tasks Are Repetitive, Manual and Laborious, and Time Consuming Typical Time Spent* (manual hours)
Server Maintenance
750
Change Migration
1600
Project & Metadata maintenance
500
Customization Deployments
200
Upgrades, Patches Application
100
3150 80 to 90% of Daily istrative Tasks Can Be Automated Saving Time and Effort *Based on sample use cases
#mstrworld
Agenda Two istrative Products for Automating Manual Processes
MicroStrategy Command Manager
MicroStrategy System Manager
Create scripts for automated metadata istration
Design workflows to automate multi-step, manual processes
#mstrworld
MicroStrategy Command Manager MicroStrategy Command Manager is a scripting tool that allows s to automate nearly all istrative and maintenance tasks. ‒ Uses an English like scripting language to convert a manual process into an automated one ‒ Executes a script automatically at a scheduled time without any human involvement or on demand
#mstrworld
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
MicroStrategy Command Manager What is a Command Manager Script? ‒ It is a collection of Command Manager Commands to carry out Intelligence Server tasks.
Where do I create Command Manager Script? ‒ In the Command Manager Application GUI.
#mstrworld
Scripting Made Easy Really?! How can scripting ever be easy?!
With Templates (called Outlines)
#mstrworld
Scripting Made Easy How do I use/apply these outlines? ‒ Simply double click or copy paste the template in the command editor ‒ Modify it to suit your requirements
‒ Check Syntax (as shown below) ‒ Execute or Save Script (to automate execution)
Outline
Command
#mstrworld
Scripting Made Easy
Auto Capitalization and Auto Color Coding of Command Words
Comments are highlighted in green
Auto Command Completion with a list of options to choose from
#mstrworld
55 Management Areas – 330+ Commands Management Area
Description
Create s and groups, and manage application access and privileges associated with each Change access properties for reports and folders for s, groups, and roles Security Management Set authorization parameters, including management of security filters Cluster Management Add or remove Intelligence Server nodes to a cluster Change the database instances associated with projects and other Application Management configuration parameters Database Connection Management Load, unload, idle, or applications for routine maintenance Event Management Import project and configuration packages List all active database connections Database Connection Management Create, alter, or disconnect specific database connections based on certain conditions Create new event-driven schedules and trigger a specific event Event Management schedule based on certain criteria Job Management Monitor and kill jobs running on Intelligence Server Object Management Create, modify, or delete objects through a script Create and alter s, groups, profiles, and addresses Subscription And Report Delivery Assign privileges to s, groups, profiles, and security roles Management Assign access control rights to projects, folders, and objects Apply security roles and filters to s and groups and Group Management
#mstrworld
Execute Scripts via Command Line Call cmdmgr.exe
Provide Log File Path
Comman d line help
#mstrworld
Provide Credentials
Provide Script Path
Encryption for added security
Save As Encrypted Script
Encrypted Script
– Sensitive information such as Intelligence Server connection information can be in the encrypted file – Encrypted scripts does not require decryption during execution – Execution Command: cmdmgr – connlessmstr –f “<encrypted script path>”
#mstrworld
MicroStrategy Command Manager Does it my OS? – Runs on all OS ed by Intelligence Server* i.e. deployable on Windows®, UNIX®, and Linux® How can I schedule the scripts I have created? – You can save the script as a Batch/Shell script and use the Operating System’s Scheduler to schedule it • For Windows, use Windows Task Scheduler • For UNIX/Linux, use Cron
– Event Based Scheduling also available
* Small subset of schema related commands are Windows-only
#mstrworld
Features: Integration with Enterprise Manager Enterprise Manager contains Out of the Box reports that generate Command Manager Scripts based on certain criteria – – – – – – – – – –
#mstrworld
Delete Unused Attributes Delete / Disable Unused s / Groups Delete Unused Reports Delete Unused Documents Delete Unused Filters Delete Unused Metrics Delete Unused Schedules Delete Unused Security Filters Delete Unused Templates Delete Unused DB Instances
Advanced Features: Use of Procedures Commands can be integrated with logic with the help of Procedures – Procedures let you harness the power of Java within Command Manager such as if-then-else analysis, looping, parameterized procedures Contains 50+ Sample Procedures – Delete Caches having low hit counts, larger than certain size etc. – Kill Jobs having low priority by – List DB s that have no dependents
#mstrworld
MicroStrategy Command Manager Overall Architecture And Workflow Of Command Manager
Scripts developed using Command Manager UI
Script Execution
Interfaces (Graphical & Command Line)
Formatting and Displaying Results MicroStrategy Open API Command Execution
Intelligence & Narrowcast Server
#mstrworld
Syntax Validation, Command Parsing, Java Compiling
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
Benefits & Summary
MicroStrategy Command Manager enables automation of tasks with no necessary human attention, saving time and money
Apart from automation, Command Manager also helps in Bulk Object Manipulations/Creation
Integration with 3rd party tools to maintain company standards
Scripts are Cross-Platform and can be executed from the command line interface.
Encryption ensures that scripts containing sensitive information can be distributed securely
Procedures extend flexibility of Command Manager by incorporating logic within the Command Manager commands
#mstrworld
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
MicroStrategy System Manager MicroStrategy System Manager is a process automating tool that enables BI teams to convert a sequential, multi-step, multi-product, manually executed process into an automated workflow. Available Processes
Workflow Designer file using FTP
MicroStrategy Processes Cloud Processes TRUE
System Processes File Operations
Success?
Execute Command Manager
Send Email to notify
ODBC Operations Exit
#mstrworld
FALSE
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
MicroStrategy System Manager – GUI Left side displays the different processes, right side has the canvas
#mstrworld
2 7
MicroStrategy System Manager – GUI Simply DRAG and DROP required process on the Canvas
#mstrworld
2 8
MicroStrategy System Manager – GUI Connect the processes using different Connectors
#mstrworld
2 9
MicroStrategy System Manager – GUI Connect the processes using different Connectors
#mstrworld
3 0
MicroStrategy System Manager Process Properties in the GUI
#mstrworld
MicroStrategy System Manager – Process Properties Each process in the workflow is defined by its properties Property Name
FTP Process
Property Value
s are Hidden
Property Value = Global Parameters
#mstrworld
MicroStrategy System Manager – Global Parameters
Export Global Parameter Parameters File
#mstrworld
3 3
MicroStrategy System Manager – Global Parameters
Export Parameter File
#mstrworld
3 4
MicroStrategy System Manager – Global Parameters Import Parameter File C:\Paramaters.smp
#mstrworld
3 5
MicroStrategy System Manager – Global Parameters Global Parameters are available to every process
$(x) Global Parameters
Workflow
Call Parameters in any Process
#mstrworld
MicroStrategy System Manager – Global Parameters Share parameters between workflows by exporting them
Export
Modify values
Export to file, modify values and re-use the Workflow in different BI Environments
Import & reuse
Workflow
BI Env 1 BI Env 2 Workflow 1
Export
Export to file and import in other Workflows #mstrworld
Import
Workflow 2
Workflow 3
BI Env 3
MicroStrategy System Manager Automated Workflows Greatly Reduce IT Personnel Time and Eliminate Manual Effort Intelligence Server Maintenance 1. 2. 3. 4.
Alert BI team Re-start Intelligence Server Monitor the system Notify BI team
310
#mstrworld
Daily Report Execution Schedule 1. 2. 3. 4.
Monitor ETL completion Publish Cubes Wait for Cubes to publish Trigger Subscription Schedules 5. Notify BI teams
540
BI Object Change Migration 1. 2. 3. 4.
Update Packages to Test system Generate rollback packages Apply changes to each project 5. Alert developers to errors 6. Notify BI Test teams
1,175
EXAMPLE – BI CHANGE MIGRATION With System Manager, Each Step Defined in a Single Workflow Runs Automatically
Step 1 Update Package Via FTP
Step 3 Step 2 Copy files Generate Into Folders Roll Back packages
With FTP Client
Step 4 Migrate Changes to the project
Step 5 Run Integrity Manager tests
With MicroStrategy Integrity Manager
With MicroStrategy Command Manager
With File copy operatio n #mstrworld
With MicroStrategy Object Manager
FAIL
Step 6 Alert BI Team
Send Email to BI Team
Send Alert to istrat or
MicroStrategy System Manager BI Change Migration Workflow in System Manager
File Operations
Programmati Programmatic cElements Elements
Command Manager Process
#mstrworld
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
MicroStrategy System Manager – Demonstration Build & Execute a System Manager Workflow for BI Change Migration and migrate a NEW DASHBOARD to PRODUCTION Send Email to BI Team
Generate Roll Back Packages With MicroStrategy Command Manager
Update Package With FTP Client
Migrate Changes With MicroStrategy Object Manager
FAIL
Alert
Instead of ONE new Dashboard, we will create a Workflow to update our BI Environment with FOUR NEW DASHBOARDS at once
#mstrworld
MicroStrategy System Manager - Processes A wide variety of processes enables creation of complex workflows MicroStrategy istrative Processes
Non - MicroStrategy Processes
Programmatic Elements
Launch/Terminate Amazon Cloud Instances
Decisions to control true/false process status Iterative Retrieval
Integrity Manager
Manage VMWare VMs and VApps
Configuration Wizard
Copy/Delete/Move Files
Command Manager Object Manager
File Counter
Project Merge
Find/Replace File Content
Define global System Manager Parameters
Project Duplicate
Unzip files
Update Parameters
Project Mover
With FTP
Server Status
Execute Application
Loops to Process Multiple iterations
Installation
Execute SQL
Connectivity Wizard
Send Email in HTML
Split/Merge Execution
Create Package
Create/Rename Files
Pause
$(x)
Node
Name Nodes and Add Comments
Combine any of these processes in any desired sequence in one workflow
#mstrworld
Execute Workflows via Command Line Call MaSysMgr.exe
Provide Workflow Path
Provide Parameter File Path
Command line help
#mstrworld
Presentation Agenda MicroStrategy Command Manager Need for & Introduction Basics & Product Features Summary
MicroStrategy System Manager Introduction Products Features Demo
Benefits & Summary
#mstrworld
Success Story – MicroStrategy Cloud Automation Process Diagram MicroStrategy Cloud
Create Environment
Customer Environment Linux Development Linux Production 1
Automation Server
Linux Production 2 Windows Utility Windows SQL Server Metadata
Notify s
Cloud #mstrworld
Cloud Teams
Success Story – MicroStrategy Cloud Project Statistics System Manager 26 Workflows System Manager Processes Code Files
195
Code Lines
9,346
Log Entries per Run
#mstrworld
501
15,000+
Benefits to Your Organization Improved Process Efficiency & Reliability Efficiency
Lowered istrative Costs
Manual Automate d
•
Less people required
Standardized processes
•
•
Reduced manual effort
Human-induced errors eliminated
•
•
No human oversight necessary
•
Training is easy
•
Better Process Tracking and Alerting
#mstrworld
Simplified BI istration
Just launch a workflow to perform any routine process!
•
Workflows run on schedule
•
De-centralization of processes
Summary
MicroStrategy System Manager Enables s to Convert Manual Processes Into Automated Workflows
Workflows can incorporate MicroStrategy & Non-MicroStrategy processes, in any sequence
Parameterization of workflows enable the same workflow to be used with different inputs
Workflows run on any OS ed by the MicroStrategy Intelligence Server & can be run via Command Line
Workflows can be scheduled to run automatically, without any human involvement or can be triggered manually as well
#mstrworld
QUESTIONS Dennis Liao Email:
[email protected]
#mstrworld