Addons Released Mscloudservices g332y

/en-US/app/Splunk_TA_microsoft-cloudservices/redirect

as your

redirect URL.

Create an application in Microsoft Azure AD Follow the instructions in the Microsoft documentation to create an active directory application: Use portal to create an Azure Active Directory application and service principal that can access resources for either your Azure portal or Azure Government portal. 23

When prompted, select or enter the following parameters. Sign-on URL and App ID URI: Required for Microsoft Office 365 . These are irrelevant for the Splunk Add-on for Microsoft Cloud Services. Enter any valid URIs. Reply URL: Required for Microsoft Office 365 . Enter the redirect URL from the step Obtain a redirect URL for your application. Client ID: Required for Microsoft Office 365 and Azure App . Copy this value. You need this value and a valid secret key to connect to your from the add-on. Key: Required for Microsoft Office 365 and Azure App . Copy this value to a secure location as soon as the Azure AD console displays it. X.509 certificate: Required for Microsoft Office 365 . Skip this section of the instructions for now. You can add this later, following the instructions in Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services. Tenant ID: Required for Azure App . Copy this value for the future use.

Add permissions to your Active Directory Applications Application permissions to access Office 365 Management APIs In order to get data from Office 365 management APIs, you need to add the Office 365 Management APIs to the permissions to other applications list. Select all the required permissions listed under both Application Permissions and Delegated Permissions: • Read activity reports for your organization • Read activity data for your organization • Read service health information for your organization

For detailed instructions, see the permissions your app requires to access the Office 365 Management APIs on MSDN.

24

Office 365 tenant consent Now that the add-on is configured with the permissions it needs to use the Office 365 Management APIs, a tenant must explicitly grant the add-on these permissions in order to access their tenant's data by using the APIs. To grant consent, the tenant must to Azure AD, using the following specially constructed URL, where they can review your add-on's requested permissions. This step is not required when using the APIs to access data from your own tenant. For detailed information, see 365 tenant consent on MSDN. Application permissions to access Windows Azure Service Management APIs Select Access Azure Service Management as organization under Delegated Permissions.

Grant the Active Directory Application Read Access Note: This is an optional procedure, you only need to perform this step if you need to configure the Azure App in Splunk Add-on for Microsoft Cloud Service. After creating the Active Directory Application, to either the Azure portal or the Azure Government portal to grant this application the read access to Microsoft Cloud Service (You must have a P1 Active Directory level edition or higher to perform this operation). See Use portal to create an Azure Active Directory application and service principal that can access resources for more information.

Connect to your Microsoft Office 365 with the Splunk Add-on for Microsoft Cloud Services Set up integration between the Splunk Add-on for Microsoft Cloud Services and your Microsoft Office 365 so that you can ingest your Microsoft cloud services data into the Splunk platform. Note: You can only connect to your using Splunk Web, configure Microsoft Office 365 via configuration file is not ed.

25

Prerequisite: Before you complete these steps, follow the directions in Configure an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft Cloud Services to prepare your Microsoft for this integration.

Connect to your using Splunk Web 1. Clear your cache, start a new browser session, or use a different browser than the one you use to sign in to the Azure AD console. This best practice helps to avoid issues with incorrectly cached credentials that interfere with the OAuth dance. 2. Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on. 3. Launch the add-on, then click Configuration > O365 . 4. Click > Add . 5. Enter a friendly Name for the . 6. Chose the type, public or GCC High if you are using the high-security government version. 7. Enter the Client ID that Azure AD automatically assigned to your integration application. 8. In the Key (Client Secret) field, enter the secret key that you created for your application in the Azure AD console. 9. Click Add. 10. The Splunk Add-on for Microsoft Cloud Services authenticates using the client ID and secret you provided. Microsoft prompts you to with your credentials to complete the authentication. Next step: Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services.

Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for 26

Microsoft Cloud Services Note: This step is only used when you need to configure Microsoft Office 365 Management APIs inputs. If you don't have to configure Microsoft Office 365 Management APIs inputs, you can skip this step. This add-on uses OAuth to authenticate from the Splunk platform to your Microsoft Office 365 using an authorization token refreshed automatically with a refresh token. This authorization token has a mandatory expiration set by Microsoft, so the refresh token only keeps your integration current for a limited period. To avoid having to periodically re-enter a secret key manually, you can a Base64-encoded X.509 Certificate and private key to enable service-to-service calls and use the key credentials to update the manifest of your integration application in Azure AD. If you are using the configuration files to configure your connection to your Microsoft cloud services, this procedure is mandatory. If you are using Splunk Web, this procedure is highly recommended. If you skip this step, then when your authorization token expires, you will need to edit your configuration that handles your connection to Microsoft Office 365 by entering a new secret key from the Azure AD console.

Configure a certificate and private key You can configure the certificate and private key in Splunk Web on your data collection node (recommended), or in the configuration files. Configure a certificate in Splunk Web 1. In Splunk Web on the instance responsible for data collection with this add-on, go to the Splunk Add-on for Microsoft Cloud Services > Configuration. 2. Click Certificate. 3. Choose one of the two options. Option 1: Using your preferred tool, generate a X.509 your own certificate certificate file and a private key with a length of at and private key least 2048 characters and them on this screen. For more information ing self-signed certificates, see How to self-sign certificates. Click Choose a Certificate and browse to the certificate file (.cer) in your file system. 27

You need to decrypt the private key before you it on Splunk add-on for Microsoft Cloud Service. Option 2: Use an Choose this option if you want to use a certificate auto-generated that the Splunk Add-on for Microsoft Cloud certificate Services auto-generates for you. 4. The add-on displays the keyCredentials JSON object for your certificate. 5. Copy the entire JSON object to your clipboard. Next, see the certificate credentials to your integration application in Azure AD. Configure a certificate using the configuration files 1. Generate a Base64-encoded X.509 certificate and put it in $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/certificate.cer.

Make sure the certificate is a X.509 certificate and the key length is at least 2048 . Shorter key lengths are not accepted by Microsoft Office 365 as valid keys. 2. Create

$SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_cert

and add the following stanza. [certificate] private_key =

3. Next, you need to obtain the keyCredentials JSON object. Run

4.

python $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/bin/splunk_ta_microsoft_office365/g $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/certificate.cer Copy the results to the manifest_json field in $SPLUNK_HOME/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_cert

Once your certificate is created using Splunk Web or using the configuration files, it will look like the following example.

"keyCredentials": [{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977", "customKeyIdentifier": "3wLEeFNsctRc+jWX057nFP0+QD8=", "value": "MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU2VsZiB "type": "AsymmetricX509Cert", "usage": ""}]

For more information ing self-signed certificates, see How to self-sign certificates. 28

Next, continue with the procedure in the next section.

the certificate credentials to your integration application in Azure AD 1. Sign in to the Azure management portal and navigate to the integration application that you created in Connect to your Microsoft Office 365 with the Splunk Add-on for Microsoft Cloud Services. 2. Click Manage Manifest > Manifest. It will look similar to the below example. {

"appId": "0399fdb3-c651-4360-ae33-97ed0598b5af", "appRoles": [], "availableToOtherTenants": false, "displayName": "zliang-test-app", "errorUrl": null, "grouphipClaims": null, "optionalClaims": null, "acceptMappedClaims": null, "homepage": "http://localhost:8000", "identifierUris": [ "https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641 ], "keyCredentials": [], "knownClientApplications": [], "Url": null, "oauth2AllowImplicitFlow": false, "oauth2AllowUrlPathMatching": false, "oauth2Permissions": [ { "ConsentDescription": "Allow the application to access zliang-test-app on behalf of the signed-in .", "ConsentDisplayName": "Access zliang-test-app", "id": "8448c8ef-a250-481e-ba5c-d877badd3e07", "isEnabled": true, "type": "", "ConsentDescription": "Allow the application to access zliang-test-app on your behalf.", "ConsentDisplayName": "Access zliang-test-app", "value": "_impersonation" } ], "oauth2RequirePostResponse": false, "objectId": "aa082da8-0f43-4a09-a364-630f4df75a62", "Credentials": [], "publicClient": false, "replyUrls": [ "http://localhost:8000"

29

], "requiredResourceAccess": [ { "resourceAppId": "00000002-0000-0000-c000-000000000000", "resourceAccess": [ { "id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6", "type": "Scope" } ] } ], "samlMetadataUrl": null }

3. Open the manifest in a text editor. 4. Place your cursor inside the empty brackets after "keyCredentials": and replace the keyCredentials key-value pair with the one generated in your configured certificate. See the below example. {

"appId": "0399fdb3-c651-4360-ae33-97ed0598b5af", "appRoles": [], "availableToOtherTenants": false, "displayName": "zliang-test-app", "errorUrl": null, "grouphipClaims": null, "optionalClaims": null, "acceptMappedClaims": null, "homepage": "http://localhost:8000", "identifierUris": [ "https://a830edad9050849NDA3079.onmicrosoft.com/6136b06e-df48-4776-82c0-424641 ], "keyCredentials": [{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977", "customKeyIdentifier": "3wLEeFNsctRc+jWX057nFP0+QD8=", "value": "MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU "type": "AsymmetricX509Cert", "usage": ""}], "knownClientApplications": [], "Url": null, "oauth2AllowImplicitFlow": false, "oauth2AllowUrlPathMatching": false, "oauth2Permissions": [ { "ConsentDescription": "Allow the application to access zliang-test-app on behalf of the signed-in .", "ConsentDisplayName": "Access zliang-test-app", "id": "8448c8ef-a250-481e-ba5c-d877badd3e07", "isEnabled": true, "type": "", "ConsentDescription": "Allow the application to access zliang-test-app on your behalf.",

30

"ConsentDisplayName": "Access zliang-test-app", "value": "_impersonation" } ], "oauth2RequirePostResponse": false, "objectId": "aa082da8-0f43-4a09-a364-630f4df75a62", "Credentials": [], "publicClient": false, "replyUrls": [ "http://localhost:8000" ], "requiredResourceAccess": [ { "resourceAppId": "00000002-0000-0000-c000-000000000000", "resourceAccess": [ { "id": "311a71cc-e848-46a1-bdf8-97ff7156d8e6", "type": "Scope" } ] } ], "samlMetadataUrl": null }

5. Check to make sure the edited JSON is valid. 6. (Optional) If the keyCredentials array in your application's manifest is not empty, copy the value, from your generated keyCredentials array, and paste it inside your existing keyCredentials array in your manifest, along with a "," in between the copied value and the existing values in order to construct a valid JSON array. See the below example.

{"keyId": "92fe4c65-9ce3-4d6d-9c76-31b511a8a977", "customKeyIdentifier": "3wLEeFNsctRc+jWX057nFP0+QD8=", "value": "MIIDETCCAfkCCG2f2svvWEwAMA0GCSqGSIb3DQEBDQUAMCQxDDAKBgNVBAMMA3NvMTEUMBIGA1UECgwLU "type": "AsymmetricX509Cert", "usage": ""}

7. Save the file. Do not change the file name. 8. In the Azure management portal, click Manifest > Manifest. 9. the edited JSON file that you just saved. 10. On the Splunk platform instance responsible for data collection for this add-on, click on Troubleshooting. If the Certificate Status says anything other than "ed and verified as valid", wait a moment and refresh the page. If the certificate is still not reported as valid, try again with a new certificate and key file.

31

Configure Office 365 Management APIs inputs for the Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft Cloud Services • Connect to your Microsoft Office 365 with the Splunk Add-on for Microsoft Cloud Services • Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files. Note: • If you want to collect audit logs for mailbox access from Exchange Online, you need to turn on mailbox audit logging in Office 365, which is not enabled by default. See Exchange audit logging. • If you configure the Office365 input for the first time, the activity log (such as Audit.Exchange, Audit.Sharepoint and Audit.AzureActivityDirectory) will subscribe the data from Microsoft side. But it will take up to 12 hours for the first content blobs to become available for that subscription in Microsoft.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input and then select Office 365 Management APIs. 3. Enter the Name, , Data and Index using information in the input parameter table below. 4. Click Add.

32

5. that data is successfully arriving by running the following search on your search head: sourcetype=ms:o365:management*

If you do not see any events, check the Troubleshooting tab on your data collection node to that your s, forwarders, and inputs are all configured successfully. See Troubleshoot the Splunk Add-on for Microsoft Cloud Services for information about enabling this dashboard on your heavy forwarder.

Configure inputs in the configuration files Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. Create

$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/splunk_ta_ms_o365_server_m

2. Add the following stanza.

[<management_input_name>] = data = index =

3. (Optional) If you want to change the data sources or polling intervals, edit the data parameter. These default values represent all the data sources currently available for collection with this add-on. Note: CurrentStatus also includes HistoricalStatus. CurrentStatus uses the interval defined here, but HistoricalStatus uses 86400 (24 hours), because Microsoft generates historical status once per day. For more information, see https://msdn.microsoft.com/EN-US/library/office/dn707386.aspx. 4. (Optional) Configure a custom index. 5. Restart your Splunk platform instance. 6. that data is successfully arriving by running the following search on your search head: sourcetype=ms:o365:management*

33

If you do not see any events, check the Troubleshooting tab on your data collection node to that your s, forwarders, and inputs are all configured successfully. See Troubleshoot the Splunk Add-on for Microsoft Cloud Services for information about enabling this dashboard on your heavy forwarder.

Input Parameters Each attribute in the following table corresponds to a field in Splunk Web.

Attribute

Corresponding field in Splunk Web

Description

management_input_name

Name

A friendly name for your input.

The Microsoft Office 365 from which you want t data.

data

Data

The Microsoft cloud services from which you want to col through the API, with intervals for data collection for eac service. The add-on automatically lists all services curre available. You can remove any or click the interval value the frequency with which the add-on polls for new data f API. Note: CurrentStatus also includes HistoricalStatus. CurrentStatus uses the interval defined here, but Histori uses 86400 (24 hours), because Microsoft generates his status once per day. For more information, see https://msdn.microsoft.com/EN-US/library/office/dn7073

index

Index

The index in which the Microsoft cloud services data sho stored. The default is main.

Configure a Storage in Microsoft Cloud Services In order to gather data from Azure Storage Table, Azure Storage Blob and Azure Virtual Machine Metrics, you need to create or configure a storage in Microsoft Azure. Prerequisite: In order to create a storage , you need a Microsoft Azure with global permissions.

34

Create and Manage Storage See Create Azure storage s for the instructions to create and manage the storage .

Configure the Storage to get data • Splunk Add-on for Microsoft Cloud Services provides two methods for you to get Azure storage table and Azure virtual machine metrics data. You can use either Access Key or Token (SAS: Shared access signature) follow the steps below. • If you want to get Azure storage blob data, besides the two methods mentioned above, you can also use None Secret to get the data without inputting key or token. Get storage access key 1. to your Azure portal or Azure Government portal 2. Select the storage you want to use 3. Copy either Key1 or Key2 of Access Key under Settings. Get storage token (SAS) 1. to your Azure portal or Azure Government portal 2. Select the storage you want to use 3. Configure Shared access signature based on the data you want to collect, such as allowed services, allowed resource types and start and expiry date/time under Settings. 4. Generate SAS and copy it to clipboard Get storage blob data without key or token 1. to your Azure portal or Azure Government portal 2. Click All resources in the menu, then select the storage that that you want to use. 3. Click Overview and then select Blobs under Services. 4. Select the container you want to configure and then click Access policy. 5. Select Container for the Access type.

35

Connect to your Azure App with Splunk Add-on for Microsoft Cloud Services Connect between the Splunk Add-on for Microsoft Cloud Services and your Azure App so that you can ingest your Microsoft cloud services data into the Splunk platform. You can configure this connection using Splunk Web on your data collection node (recommended), or using the configuration files. Prerequisite: Before you complete these steps, follow the directions in Configure an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft Cloud Services to prepare your Microsoft for this integration.

Connect to your using Splunk Web Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on. 1. Launch the add-on, then click Configuration. 2. Click Azure App > Add Azure App . 3. Enter a friendly Name for the . 4. Enter the Client ID , Key (Client Secret) and Tenant ID using the parameter table below. 5. Click Add.

Connect to your using configuration files If you do not have access to Splunk Web on your data collection node, you can configure the connection to your using the configuration files. 1. Create or open

$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_azure_s.

2. Add the following stanza. [ <_stanza_name rel="nofollow"> ] client_id = client_secret = tenant_id =

Attributes Attribute

Corresponding name in Splunk Web 36

Description

Name

Enter a friendly name for your Azure app

client_id

Client ID

Use the Clinet ID that Azure AD automatically assigned to your integration application

client_secret

Key (Client Secret)

Enter the for Client ID

Tenant ID

Enter the Tenant ID when you Create an application in Microsoft Azure AD

_stanza_name

tenant_id

Configure Azure Audit Modular inputs for the Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft Cloud Services • Connect to your Azure App with Splunk Add-on for Microsoft Cloud Services Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input and then select Azure Audit. 3. Enter the Name, Azure , Subscription ID, Start Time, Interval and Index using the information in input parameter table below.

37

Configure inputs using configuration file Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. Create a file named mscs_azure_audit_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.


[ ] = subscription_id = start_time = index = interval =

3. Save and restart Splunk platform.


Attribute


Description

input_stanza_name

Name


Azure

The Azure App from which you want to gather data.

subscription_id

Subscription ID

The instance queries the management events belong to this subscription. The subscription ID is the one you configured in Microsoft

start_time

Start Time

The add-on starts collecting data with a date later than this time. The format is YYYY-DD-MMThh:mm:ssTZD and the default is 30 days before the configuration, e.g. 2016-07-15T09:00:00+0800 stands for fetching data from 2016-07-15 09:00:00 in UTC+8 time zone. 38

Note: The maximum start time of Azure Audit inputs is 90 days before the configuration. interval

Interval

The number of seconds to wait before the Splunk platform runs the command again. The default is 3600 seconds.

index

Index

The index in which to store Azure audit data.

Configure Azure Resource Modular inputs for the Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure an Active Directory Application in Azure AD for the Splunk Add-on for Microsoft Cloud Services • Connect to your Azure App with Splunk Add-on for Microsoft Cloud Services Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input and then select Azure Resource. 3. Fill out the Name, Azure App , Subscription ID, Resource Type, Resource Group List, Interval and Index fields using the input parameter table below. 4. Click Add.

39

Configure inputs using configuration file Configure your inputs using the configuration files on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. Create a file called mscs_azure_resource_inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.


[ ] = subscription_id = resource_type = resource_group_list = index = interval =


Input Parameter Attributes

Corresponding Fields in Splunk Web

Description

input_stanza_name

Name


Azure App

The Azure App from which you want to gather data.

subscription_id

Subscription ID

The instance queries the management events belong to this subscription. The subscription ID is the one you configured in Microsoft

resource_type

Resource Type

You can choose from Virtual Machine, Public IP Address, Network Interface Card and Virtual Network using Splunk Web, or set resource_type to virtual_machine, public_ip_address, network_interface_card or virtual_network in the configuration 40

file.

Resource Group List

The resource group list is defined by subscription ID and resource type. If you leave this field blank, this add-on will query all resource lists under the subscription ID and the resource type you choose. You can add multiple resource group list separated by commas.

interval

Interval


index

Index

The index in which the Microsoft cloud services data should be stored.

resource_group_list

Connect to your Azure Storage with the Splunk Add-on for Microsoft Cloud Services Connect the Splunk Add-on for Microsoft Cloud Services and your Azure Storage so that you can ingest your Azure storage table, Azure storage bolb and Azura virtual machine metrics data into the Splunk platform. You can configure this connection using Splunk Web on your data collection node (recommended), or using the configuration files. Prerequisite: Before you complete these steps, follow the directions in Configure a Storage in Microsoft Cloud Service to prepare your Microsoft for this integration.

Connect to your using Splunk Web Access Splunk Web on the node of your Splunk platform installation that collects data for this add-on. 1. Launch the add-on, then click Configuration. 2. Click Azure Azure Storage and enter the corresponding fields using the input parameter table below.

41

Note: There are three Secret Type that you can select to configure Azure storage , Access Key, Token and None Secret. • If you want to collect Azure storage table Azure virtual machine metrics data, you have to configure the with Access Key or Token type. • If you want to collect Azure storage blob data, you can use any of three types.

Connect to your using configuration files If you do not have access to Splunk Web on your data collection node, you can configure the connection to your using the configuration files. 1. Create or open

$SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local/mscs_storage_s.conf

2.Add the following stanza.

[ <_stanza_name rel="nofollow"> ] _name = _secret = _secret_type =


Attribute


Description

Name

The name for the storage .

_secret

Secret

You can enter the key or token generated when you Configure a Storage in Microsoft Cloud Service.

_secret_type

Access Key, If you set _secret_type=0, it Token or means the storage use None Secret None Secret type. You do not have to set Name and Secret. If you configure the inputs using configuration file, you can leave

_name

42

_name

and _secret

blank. If you set _secret_type=1, it means the storage use Access Key type. You have to enter the key generated when you Configure a Storage in Microsoft Cloud Service If you set _secret_type=2, it means the storage use Token type. You have to enter the token generated when you Configure a Storage in Microsoft Cloud Service

Configure Azure Storage Table Modular Input for Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure a Storage in Microsoft Cloud Service • Connect to your Azure Storage with the Splunk Add-on for Microsoft Cloud Services Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input. 3. Fill out the Name, Azure Storage , Table List, Start Time, Interval, Index and Sourcetype fields using the input parameter table below. 43

Configure inputs using configuration file 1. Create a file called inputs.conf under $SPLUNK_HOME/etc/appes/Splunk_TA_microsoft-cloudservices/local

2. Configure Azure storage table inputs with the following stanza.

[mscs_storage_table:// ] = table_list = start_time = index = interval = sourcetype =


Input Parameters Attributes


Descr

mscs_storage_table://

Name


Azure Storage

Choose a Storage you have con

Table List

The table list under the storage . separated by commas. You can also use If the table name uses regex syntax, plea name. For example: table*, :table\d+.

start_time

Start Time

The add-on starts collecting data with a d 30 days before the configuration. The for e.g. 2016-07-15T09:00:00+0800 stands 09:00:00 in UTC+8 time zone.

collection_interval

Interval

The number of seconds to wait before th again. The default is 3600 seconds.

index

Index

The index in which to store Azure Storag

sourcetype

Sourcetype

The default is mscs:storage:table.

table_list

Note: If you want to change the default s field of the event, the behaviour is unpred the timestamp field. To prevent the issue 44

under

SPLUNK_HOME/etc/apps/Splunk_TA_micro

Configure Azure Storage Blob Modular Input for Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure a Storage in Microsoft Cloud Service • Connect to your Azure Storage with the Splunk Add-on for Microsoft Cloud Services Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files. Note: Since the format of the data in Azure Storage Blob channel varies (including text and binary data), Splunk suggests you to leverage the options for sourcetypes to make the event data more effective. See Overview of Event Processing for details.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input and select Azure Storage Blob. 3. Enter the Name, Storage , Container Name, Blob list, Interval, Index and Sourcetype using the inputs parameters table below.

Configure inputs using Configuration File 1. Create a file called inputs.conf under $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/local.

2. Configure Azure Storage Blob input with the following stanza.

45

[<mscs_storage_blob:// >] = container_name = blob_list = exclude_blob_list = blob_mode = decoding = index = interval = sourcetype =

Inputs Parameter Each attribute in the following table corresponds to a field in Splunk Web.

Attribute


Description

mscs_storage_blob://

Name

Enter a friendly name of your inputs.

Azure Storage

Select the storage name you configured.

container_name

Enter the container name under the storage Container Name . You can only add one container name for each input.

blob_list

Blob List

Enter the Blob name which you want to collect the data from. You can add multiple blob names separated by commas. If you leave this field empty, this add-on will collect all the blob lists under the Container Name you just configured. Note: You can enter the specific blob list name, use wildcard or use

46

regex expression in this field. • If you want to collect data from a specific blob list, just enter the name of the blob list here, such as blob_name. • You can use wildcard in this field, e.g. blob*, this add-on will collect data from the blob lists of the names starting from blob. And you can also use comma to separate multiple blob names, e.g. blob, name*. • If you want to use regex, the syntax is JSON format: {"regex syntax":3}, 3 stands for regex file. • If you want to enter the blob list which has both wildcard and regex, you can enter both separated by commas, for example, {"regex syntax" :3, blob* :2}, 2 stands for 47

wildcard list. • If you want to enter the blob list using all of the three expressions, you can use the syntax like {"regex syntax" :3, blob* :2, blob :1}, 1 stands for using a specific blob list name. Note: • The blob name must be at least one character long but cannot be more than 1,024 characters. • Blob names are case-sensitive. • Reserved URL characters must be properly escaped. • The number of path segments comprising the blob name cannot exceed 254. exclude_blob_list

Excluded Blob List

48

Optional. Enter the Blob name that you do not want to collect the data from. You can add multiple blob names separated by commas. The syntax of the

Excluded Blob List is the same as Blob List. NULL

The default is append. Do not change the value of this field.

Decoding

Specify the character set of the file, such as UTF-8 or UTF-32. If you leave this field blank, this add-on will use the default character set of the file.

Interval


index

Index

The index in which to store Azure Storage Blob data.

sourcetype

Sourcetype

blob_mode

decoding

collection_interval

The default is mscs:storage:blob.

Note: If there is a file match the syntax both in Blob List and Exclude Blob List, Exclude Blob List is in higher priority. For example, if there is a blob list name blob1, and it match the syntax you set in Blob List and Exclude Blob List, this add-on will exclude this list because Exclude Blob List is in higher priority.

Configure Azure Virtual Machine Metrics Modular Input for Splunk Add-on for Microsoft Cloud Services Prerequisites: Before you enable inputs, complete the previous steps in the configuration process: • Configure a Storage in Microsoft Cloud Service • Connect to your Azure Storage with the Splunk Add-on for Microsoft Cloud Services 49

Configure your inputs on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. You can configure inputs using Splunk Web (recommended) or using the configuration files.

Configure inputs using Splunk Web Configure your inputs using Splunk Web on the Splunk platform instance responsible for collecting data for this add-on, usually a heavy forwarder. 1. In the Splunk Add-on for Microsoft Cloud Services, click Inputs. 2. Click Create New Input and select Azure Virtual Machine Metrics. 3. Type the Name, Storage , Start Time and Indexusing the input parameters below and then click Add. Note: You cannot configure Table List, Interval and Sourcetype using Splunk Web.

Configure inputs using configuration file 1. Create a file called inputs.conf under $SPLUNK_HOME/etc/appes/Splunk_TA_microsoft-cloudservices/local

2. Configure Azure virtual machine metrics inputs with the following stanza.

[<mscs_storage_table:// ] = table_list = WetricsPT1M* start_time = index = interval = 60 sourcetype = mscs:vm:metrics


Input Parameters Attribute mscs_storage_table://

Corresponding field in Splunk Web Name

50

Description A friendly name for your input.

table_list

Azure Storage

Table List

Choose a Storage you have configured. Enter a table list name under the storage . You cannot change the Table List name in Splunk Web, which is WetricsPT1M*. Note: The best practice is to keep the default value WASMetricsPT1M* in the table list.

start_time

collection_interval

Start Time

Interval

The add-on starts collecting data with a date later than this time. The default is 30 days before the configuration. The format is ?YYYY-DD-MMThh:mm:ssTZD, e.g. 2016-07-15T09:00:00+0800 stands for fetching data from 2016-07-15 09:00:00 in UTC+8 time zone. The number of seconds to wait before the Splunk platform runs the command again. The default is 60 seconds and you cannot change it in Splunk Web. If you want to change the interval time, you have to configure it using the configuration file. Note: If you want to use ITSI data model, the best practice is to set the interval to 60 seconds.

index

Index

sourcetype

Sourcetype

51

The index in which to store Azure Storage Table data. The default is mscs:vm:metrics. You cannot change the sourcetype in Splunk Web. If you want to change the sourcetype, you have to configure it using the

configuration file.

Troubleshoot the Splunk Add-on for Microsoft Cloud Services General troubleshooting For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see and resource links for add-ons in Splunk Add-ons.

Troubleshooting dashboard for Office365 inputs The Splunk Add-on for Microsoft Cloud Services includes a troubleshooting dashboard to help you identify issues with inputs, s, and X.509 certificates. Access this dashboard on your data collection node, usually a heavy forwarder. Note: The troubleshooting dashboard is only used for identifying issues of Office365 inputs, s and X.509 certificates. You cannot identify the issues for Azure related inputs and s using this dashboard. To enable this dashboard on a heavy forwarder, backup your _internal data locally by editing your local inputs.conf and outputs.conf files as shown: In local/inputs.conf

[monitor://$SPLUNK_HOME/var/log/splunk] index = _internal _INDEX_AND_FORWARD_ROUTING = true In local/outputs.conf

[indexAndForward] index=true selectiveIndexing=true

Access the dashboard by opening the app on your data collection node. Click on any error states to drill down into detailed error messages and mitigation suggestions.

52

Certificate status messages The troubleshooting dashboard and the Certificate configuration screen display a status message that describes the current state of your service-to-service call configuration. The table lists the possible status messages with more information. Status message

Description

No certificate configured yet

No certificate has been ed or auto-generated on the Certificate tab yet.

Certificate is ed but not verified yet

A certificate file and private key are ed on the Certificate tab of the add-on, but the add-on has not yet validated it against the manifest in Azure AD.

ed and verified as valid

A certificate file and private key are ed on the Certificate tab of the add-on, and the manifest in Azure AD contains the matching keyCredentials object. You have ed a certificate file and private key on the Certificate tab of the add-on, but either the certificate file or key are invalid or the manifest in Azure AD does not contain the correct keyCredentials JSON object that matches this certificate and key.

ed but invalid

• If you have not yet updated the manifest, follow the documented instructions in Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services to do so. • If you updated the manifest already but still see this message, check to make sure that your JSON formatting is valid. If the message persists, generate and a new certificate and private key and re-update the manifest.

The certificate file has been auto-generated on the Auto-generated but Certificate tab of the add-on, but the add-on has not yet not verified yet validated it against the manifest in Azure AD. The certificate file has been auto-generated on the Auto-generated and Certificate tab of the add-on, and the manifest in Azure AD verified as valid contains the matching keyCredentials object. Auto-generated but You have auto-generated a certificate on the Certificate invalid tab of the add-on, but it is either invalid or the manifest in

53

Azure AD does not contain the correct keyCredentials JSON object that matches this auto-generated certificate. • If you have not yet updated the manifest, follow the documented instructions in Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services to do so. • If you updated the manifest already but still see this message, check to make sure that your JSON formatting is valid. If the message persists, generate a new certificate and re-update the manifest.

Certificate Failed error If you receive SSL untrusted certificate error, it means that the website is not in the trusted list. Add the following website to the trusted domains. Office 365: https://manage.office.com https://.windows.net Azure and Storage related: https://management.azure.com https://*.table.core.windows.net https://*.blob.core.windows.net

Accessing Logs of Azure Inputs There are four different logs for different types of inputs. The table below is the detailed description of each log. Log Filename

Sourcetype

splunk_ta_microsoft-cloudservices_azure_audit.log

mscs:azure:audit:log

Descriptio

Azure Aud Log chann related log

Azure Resource splunk_ta_microsoft-cloudservices_azure_resource.log mscs:azure:resource:log channel related log splunk_ta_microsoft-cloudservices_storage_blob.log

54

mscs:storage:blob:log

Azure Storage

Blob channel related log

Azure Storage Table channel related log and splunk_ta_microsoft-cloudservices_storage_table.log

mscs:storage:table:log

Azure Virtual Machine Metrics channel related log

Not receiving MSCS data after configuring certificate If you are not receiving data, and your configured certificate says "Auto-generated and verified as valid", version 2.1 or above of the Splunk Add-on for Microsoft Cloud Services.

55

Troubleshooting Troubleshoot the Splunk Add-on for Microsoft Cloud Services General troubleshooting For helpful troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in Splunk Add-ons. For additional resources, see and resource links for add-ons in Splunk Add-ons.

Troubleshooting dashboard for Office365 inputs The Splunk Add-on for Microsoft Cloud Services includes a troubleshooting dashboard to help you identify issues with inputs, s, and X.509 certificates. Access this dashboard on your data collection node, usually a heavy forwarder. Note: The troubleshooting dashboard is only used for identifying issues of Office365 inputs, s and X.509 certificates. You cannot identify the issues for Azure related inputs and s using this dashboard. To enable this dashboard on a heavy forwarder, backup your _internal data locally by editing your local inputs.conf and outputs.conf files as shown: In local/inputs.conf

[monitor://$SPLUNK_HOME/var/log/splunk] index = _internal _INDEX_AND_FORWARD_ROUTING = true In local/outputs.conf

[indexAndForward] index=true selectiveIndexing=true

Access the dashboard by opening the app on your data collection node. Click on any error states to drill down into detailed error messages and mitigation suggestions.

56

Certificate status messages The troubleshooting dashboard and the Certificate configuration screen display a status message that describes the current state of your service-to-service call configuration. The table lists the possible status messages with more information. Status message

Description

No certificate configured yet

No certificate has been ed or auto-generated on the Certificate tab yet.

Certificate is ed but not verified yet

A certificate file and private key are ed on the Certificate tab of the add-on, but the add-on has not yet validated it against the manifest in Azure AD.

ed and verified as valid

A certificate file and private key are ed on the Certificate tab of the add-on, and the manifest in Azure AD contains the matching keyCredentials object. You have ed a certificate file and private key on the Certificate tab of the add-on, but either the certificate file or key are invalid or the manifest in Azure AD does not contain the correct keyCredentials JSON object that matches this certificate and key.

ed but invalid

• If you have not yet updated the manifest, follow the documented instructions in Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services to do so. • If you updated the manifest already but still see this message, check to make sure that your JSON formatting is valid. If the message persists, generate and a new certificate and private key and re-update the manifest.

The certificate file has been auto-generated on the Auto-generated but Certificate tab of the add-on, but the add-on has not yet not verified yet validated it against the manifest in Azure AD. The certificate file has been auto-generated on the Auto-generated and Certificate tab of the add-on, and the manifest in Azure AD verified as valid contains the matching keyCredentials object. Auto-generated but You have auto-generated a certificate on the Certificate invalid tab of the add-on, but it is either invalid or the manifest in

57

Azure AD does not contain the correct keyCredentials JSON object that matches this auto-generated certificate. • If you have not yet updated the manifest, follow the documented instructions in Configure a certificate and private key to enable service-to-service calls for the Splunk Add-on for Microsoft Cloud Services to do so. • If you updated the manifest already but still see this message, check to make sure that your JSON formatting is valid. If the message persists, generate a new certificate and re-update the manifest.

Certificate Failed error If you receive SSL untrusted certificate error, it means that the website is not in the trusted list. Add the following website to the trusted domains. Office 365: https://manage.office.com https://.windows.net Azure and Storage related: https://management.azure.com https://*.table.core.windows.net https://*.blob.core.windows.net

Accessing Logs of Azure Inputs There are four different logs for different types of inputs. The table below is the detailed description of each log. Log Filename

Sourcetype

splunk_ta_microsoft-cloudservices_azure_audit.log

mscs:azure:audit:log

Descriptio

Azure Aud Log chann related log

Azure Resource splunk_ta_microsoft-cloudservices_azure_resource.log mscs:azure:resource:log channel related log splunk_ta_microsoft-cloudservices_storage_blob.log

58

mscs:storage:blob:log

Azure Storage

Blob channel related log

Azure Storage Table channel related log and splunk_ta_microsoft-cloudservices_storage_table.log

mscs:storage:table:log

Azure Virtual Machine Metrics channel related log

Not receiving MSCS data after configuring certificate If you are not receiving data, and your configured certificate says "Auto-generated and verified as valid", version 2.1 or above of the Splunk Add-on for Microsoft Cloud Services.

59

Reference Lookups for the Splunk Add-on for Microsoft Cloud Services The Splunk Add-on for Microsoft Cloud Service has the following lookups that map fields from Microsoft Cloud Service systems to CIM-compliant values in the Splunk platform. The lookup files are located in $SPLUNK_HOME/etc/apps/Splunk_TA_microsoft-cloudservices/lookups. Filename

Description Maps a status field to a friendly description.

o365_certficate_status_lookup.csv

Maps o365_management_api_data_lookup.csv

management_api_data

field to a friendly name. Maps Operation and ResultStatus to model_type,action, change_type, and

o365_model_lookup.csv

object_category

fields.

o365_model_operation_only_lookup.csv

Maps Operation to model_type, action, change_type, and object_category

fields. Maps ResultStatus to a CIM-compliant status value.

o365_status_lookup.csv

o365_troubleshooting_error_code_lookup.csv

60

Maps o365_error to Problem, Problem Detail,Possible Reason, and Proposal values for the Troubleshooting

dashboard. Maps microsoft_error_code to o365_error, Problem, Problem o365_troubleshooting_microsoft_error_code_lookup.csv Detail, Possible Reason, and Proposalvalues for

the Troubleshooting dashboard. Maps vm_size to u_cores, mem_capacity and

mscs_vm_u_mem_storage.csv

storage_capacity.

mscs_vm_ip.csv

Maps vm_id to private_ip and public_ip.

mscs_vm_power_state.csv

Maps a power_state field to a common description.

Performance reference for the Azure storage input in the Splunk Add-on for Microsoft Cloud Services This page provides reference information about Splunk's performance testing of the Azure storage input in Splunk Add-on for Microsoft Cloud Services. The testing occurred with version 2.0.0, when the Azure storage input was first introduced. Use this information to enhance the performance of your own Azure storage data collection tasks. Note: Many factors impact performance results, including file size, file compression, event size, deployment architecture, and hardware. These results represent reference information and do not represent performance in all environments.

Testing Architecture Splunk tested the performance of the Storage input using a single-instance Splunk Enterprise 6.4.3 on an C4 High-U Double Extra Large instance to ensure U, memory, storage, and network do not introduce any bottlenecks. 61

Instance specs: Note: The EC2 in the testing environment is in the same area of Azure storage input, the network latency is low. Instance type

C4 High-U Double Extra Large

Memory

15 GB

Compute Units (ECU)

31 Unit

Cores

8

Storage Type

GP2(SSD)

Architecture

64-bit

Network performance

High

EBS Optimized: Max Bandwidth 1000 Mbps

Test Environment Deployment Type

Role

EC2 Type

Count

Standalone Deployment Standalone C4 High-U Double Extra Large 1

Testing Result Testing Result for Azure Storage Table inputs The detailed test result is listed below. • Input number stands for the number of the inputs, one input collects one table. • Each table contains 2,131,200 events. • Each event is 500 Bytes. Input Number

Throughput (MB/s)

Throughput (GB/day)

Throughput (Event/s)

1

3.44

290

7045

2

5.7

480

11670

4

6.84

577

14000

8 6.12 516 12533 Analysis: The maximum throughput is 6.8MB/s with four data inputs. 100% higher than one data input. It can scale 100% with max throughput 577GB/day 62

for single instance. Testing Result for Azure Storage Blob inputs The detailed test result is listed below. • Each Blob input contains 259,202 events. • Each event is 190 Bytes. Blob Number

Blob Size

Input Number

Throughput (MB/s)

Throughput (GB/day)

Throughput (Event/s)

20

50 M

1

3.8 MB/s

320

21136

40

50 M

2

7.7 MB/s

650

42092

80

50 M

4

12.5 MB/s

1054

72960

160 50 M 8 12.1 MB/s 1020 66500 Analysis: If we use large file size (50M in our test environment), the max throughput is 12.5MB/s with four data inputs, about 300% higher than the single input.

APIs used in the Splunk Add-on for Microsoft Cloud Services The following table below lists the APIs that are used in the Splunk Add-on for Microsoft Cloud Services. Inputs Name Azure Storage Table

Method Query Tables

Description (Link to Microsoft site) https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx

Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx

Azure List Blobs Storage Blob Get Blob

https://msdn.microsoft.com/en-us/library/azure/dd135734.aspx https://msdn.microsoft.com/en-us/library/azure/dd179440.aspx

Azure Audit

Azure Insights - List the https://msdn.microsoft.com/en-us/library/azure/dn931934.aspx management events

Azure Resource -

List all virtual machines in a

https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/

63

Virtual Machine

resource group

Azure Resource Public IP Address

List public IP addresses https://msdn.microsoft.com/en-us/library/azure/mt163657.aspx within a resource group

Azure Resource Network Interface Card

List network interface cards https://msdn.microsoft.com/en-us/library/azure/mt163627.aspx within a resource group.

Azure Resource Virtual Network

List virtual networks https://msdn.microsoft.com/en-us/library/azure/mt163587.aspx within a resource group

Get the instance view of a virtual machine

https://docs.microsoft.com/en-us/rest/api/compute/virtualmachines/

Azure Virtual Query Tables https://msdn.microsoft.com/en-us/library/azure/dd179405.aspx Machine Query Entities https://msdn.microsoft.com/en-us/library/azure/dd179421.aspx Metrics

Tenant-specific content blobs Office 365 Management https://msdn.microsoft.com/en-us/office-365/office-365-manageme Activity DLP.All is not ed

64

Addons Released Mscloudservices g332y

Overview 26281t

More details 6y5l6z

Related Documents 3h463d

Addons Released Mscloudservices g332y

Addons Ipcop 3x6s5y

Chemistry Released 6jh6n

Inkscape Extensions And Addons 2n4j5w

Chart Nexus Addons Manual 1n68

Mccabe Emails Released 4l39e

More Documents from "r" 34682y

Maquina Fusion Ppr 2.pdf 3g6x1w

4p345f

Impulsos Primarios Y Estimulos Motivantes 3r2k45

Lidar Geomecanics 2h3n6

Bianchi 6z6v34

Cara Cetak Foto Agar Hemat Kertas Akal_akalan_tukang_cetak_foto_sialan r81n