This article includes links to and information from outside resources. We are not responsible for the content on the external site. If there is a link that does not work, please email [email protected] and we will attempt to adjust the information with an updated link.
What is Dataverse Storage?
The Dataverse, by Microsoft, is a cloud-based storage space that organizations use to store business application data securely. Data in Dataverse is organized as tables, each representing a specific collection of records. This includes out-of-the-box tables, such as activities, contacts, accounts (Relationships in Engage), etc., built into applications and custom tables that reflect entities specific to an organization.
The Dataverse provides three types of cloud capacity where Microsoft tracks consumption in the Power Platform Admin Center:
-
Database Capacity - transactional database storage for entity definitions and record data.
-
File Capacity - storing attachments on emails and notes in Engage apps and Power Apps, including images, PDFs, documents, and spreadsheets.
-
Log Capacity - audit logs tracking record and attribute data changes, including field value updates, security role updates, and date/time of access.
How can I Access Dataverse Storage?
Storage can be accessed through the Microsoft Power Platform Admin Center at admin.powerplatform.microsoft.com.
Who can access Power Platform?
-
Environment Admin - can view reports for the environments to which the Admin has access.
-
Power Platform admin – can view reports for all environments.
-
Dynamics 365 admin - can view reports for all environments.
-
Microsoft 365 Global admin – can view reports for all environments
How do I review the Dataverse Capacity?
Note: If you have received a warning message from Microsoft regarding your capacity threshold or want to review how much data your environments are consuming, follow the steps below.
1. Login to the Microsoft Power Platform Admin Center at admin.powerplatform.microsoft.com.
2. Navigate to Resources>Capacity>Summary. In the Summary Tab, you will see a Dashboard Overview of all your Environments.
Only the following Administrative Roles are permissioned to see the Summary tab:
-
Tenant Administrator
-
Power Platform Administrator
-
Dynamics 365 Administrator
3. To see a more detailed environment review, click the Dataverse Tab.
4. Once you click on Dataverse Tab, this will bring you to a more detailed Dashboard where you can see all your environments and usage consumption by Categories (Database, File, Log); you can export the Dashboard by CSV, and if you want to see more details by Environment click on the icon under Environment List column to deep dive into a specific environment.
If you have clicked the more details icon, the following Dashboard will open detailing the Top Table with the most Data. If you want to review all tables, you can click on the hamburger icon to download them and their consumption in a CSV format.
How Can I Manage Capacity?
When you are approaching your capacity threshold or are over capacity in any of these storages, you will receive an email from Microsoft with more detailed information. The email will be sent to all System Administrators in your firm.
Capacity Thresholds can be managed at https://admin.powerplatform.microsoft.com/ >Settings>Storage Consumption Warning Threshold.
Best Practices for Managing Storage Capacity
1. Optimize Components in Dataverse - Database Usage
When trying to reduce storage capacity and optimize components in Dataverse, MSFT recommends the following methods documented in this article Free up storage space - Power Platform | Microsoft Learn. However, after reviewing, Salentica ONLY recommends Method 7, Method 8, and Method 9 - all three Methods affect the Database space allocation.
2. Reduce the AsyncOperations Base Table
Salentica recommends that you regularly review and purge your AsyncOperations Base table to help reduce the Dataverse consumption. We recommend reviewing this on a monthly or quarterly basis to optimize your storage.
AsynchOperation Base Table
What is the AsyncOperation Base Table? The asynchronous service executes long-running operations independent of the main Dynamics 365 for Customer Engagement core operation. This results in improved overall system performance and improved scalability. The asynchronous service features a managed queue for the execution of asynchronous registered plug-ins, workflows, and operations such as bulk mail, bulk import, and campaign activity propagation. These operations are registered with the asynchronous service and executed periodically when the service processes its queue.
Best Practice: Reduce your AsynchOperation Table. You can try reducing data in this Table by deleting Completed, Cancelled, and Failed Jobs older than 30 days. More information can be found here.
Note: When creating the Recurring Job, ensure ONLY the following Status Reason is selected Succeeded, Cancelled, and Failed. Also, before deleting, selecting Preview Records to view is always recommended to ensure the correct data will be deleted.
The above job was executed in an environment with many failed Workflows, and we were able to recover some additional space.
If you suspect, there is a vast amount of data to be deleted, 50k or more; it might be a good idea to schedule the Bulk Edit Job to run after hours.
3. Monitor the Files (Attachments) Uploaded to Engage
Attachments can be uploaded to Engage either intentionally or unintentionally. Users can attach documents to Activities and upload to Notes - ‘intentional upload’ but the ‘unintentional upload’ would occur when users track email to the CRM, and there are images embedded in the email, including logos or other embedded files.
For more details about ‘allowed’ file types and sizes, visit your Power Platform admin center in the Environment (choose the environment, i.e. Production or Sandbox) → Product.
Default Attachment Size - navigate to Environments > [choose org] > Settings > Email Settings and review the Attachments default size below. This setting can be adjusted up or down.
Attachments
To mitigate the issue of file growth that cannot be adjusted via the above options, There is a third-party tool that can be used to extract Attachments and preserve them in a separate storage cloud. Attachment Extractor can reduce the capacity used for Attachments. Find more information at AttachmentExtractor for Microsoft Dynamics 365.
WebResourceBase Table (File)
This Table contains all the files used in Web development; this can include images for your entities, views, javascript, etc. Web resources provide client-side components that provide custom user interface elements.
With Microsoft performing weekly updates in this CRM environment, orphan components are left behind in the tables taking space that is no longer needed. Luckily Microsoft has scripts to optimize this Table and remove orphan objects. If you need assistance with this Table, please log a ticket via our Help Center, and our team will start the process with Microsoft.
4. Monitor Audit History tracked for Log Usage
AuditBase
This is where all your Audit data is stored - click here to read more about Audit; At the same time, it might be a good idea to keep track of all changes; over time, these changes can grow exponentially, so it is essential to review and determine what Entities (Table) and Attributes (Columns) you want to keep auditing.
How do I view what I am Auditing?
The fastest and simplest way to achieve this is by:
-
Navigate to https://admin.powerplatform.microsoft.com/
-
Click Environments and select the Environment you are analyzing
-
Click on Settings (located in the top ribbon)
-
Expand Audit and Logs and click on Audit Summary View. From the View, you can click on the filter icon, then click on Entities; the items shown in the drop-down are the entities you have Audit enabled.
You can also use Audit Center provided by Home · XrmToolBox to see and manage audit settings from one spot.
XRMToolBox is a free tool developed by the CRM Community and not owned by Salentica. Salentica does not troubleshoot or is responsible for the outcome of using this tool.
To enable/disable Audit, please review Audit in Engage.
What does Salentica recommend?
- We recommend disabling Audit on the following fields - these fields are updated daily by Data Broker.
-
Review your Audit retention; more information can be found here.
-
Delete Audit if it is no longer needed. How to Delete Audit Logs Manage Dataverse auditing - Power Platform.
Note:
-
Need to backup your Audit before deletion? You can use “Audit History Extractor” provided by
Home · XrmToolBox to export the Audit in CSV files, or you can create a ticket with us via our Help Center and our team could review and advise you on the next steps.
-
Deleting log files or any data in Salentica Engage is considered a hard delete and cannot be undone. This will be a billable effort if data is deleted and you need it recovered.
When Microsoft reviews your storage allocation, they also consider Sandbox and this Environment is usually a good starting point to reduce capacity. If your Sandbox environment is or was a copy of Production, chances are you might have more data than what you need, and you will need to evaluate if you still need this data. If the answer is NO, then data can be deleted from this Environment, and the easiest way to accomplish this is by refreshing your Sandbox with Schema Only. More information can be found here. Lastly, review your Sandbox environment and see if you also have Audit enabled; if it is and you don’t have a business reason to keep it, then it is recommended to deactivate the Audit and delete the data (if possible).
After you have Optimized your Environment, you will need to wait for at least 24 hours to see if there are any positive changes, you will likely need to return to the main Dashboard to review, and if you still need more space, the only additional alternative is to buy more capacity for this; please submit a ticket in our Help Center.
FAQ
Q. I deleted data, but why am I not seeing the difference in the Capacity Dashboards?
A1. The effects of deleted data will not be visible immediately; allow at least 24hrs for the dashboards to refresh
A2. The data that was deleted might not make a big impact in reducing the storage capacity.
Q. When I review my Dashboard View, I see other tables not mentioned in this document; why is that?
A. Salentica Engage has many tables, and it will be challenging to review all of them; this article was written to review the top culprits. However, if you need to spend time with a member of our team to work with you and analyze the Table, please create a ticket in our Help Center.
Q. Can I change the threshold alerts sent by Microsoft?
A. Yes, the threshold alert can be changed in Power Platform admin center >Settings>Storage Consumption Warning Threshold
Q. I need more storage space; how can I purchase more?
A. Log a ticket in Salentica’s Help Center and let us know how much more space you need, and we will process your request. If you need help to determine how much more is required, we can help with this too.
Q. If I need extra storage purchased, how much does it cost?
A. The below Table displays the cost to purchase storage for Database, File and Log Capacity.
Q. I reviewed my storage capacity, and I have exceeded my allotment. Does this mean my Environment will be shut down, or will it “stop” working?
A. No, your Environment will not be shut down, and your Salentica Engage will continue to work. However, there are downsides when you are over capacity, as you cannot create, restore or copy new environments. Also, being over capacity, you might experience performance issues. While being over capacity is not something you need to tackle immediately, it does need to be addressed as you are not in compliance with Microsoft.
For more information about the How to Administer the Power Platform Admin Center, be sure to visit this site (and bookmark it). For information specific to Dataverse Capacity, bookmark this site.
