Azure Blob Storage: How to Upload, Secure, and Manage Your Data

Accessing and Securing Azure Blob Storage

Security is paramount in enterprise storage systems. There are various ways to securely access Azure Blob Storage, such as SAS tokens, access keys, and enterprise accounts. 

Access control with SAS tokens

In production environments, you restrict access to files and data stored in Blob Storage. You generally don’t want it to be publicly accessible. Shared Access Signatures (SAS) offer a secure method of managing access to your Blob Storage data. 

SAS works via tokens, which are appended to the URL of the resource you want to share. The token includes information on the type of permission (read, write, delete, etc.), the validity of the token, and the protocols (HTTP and/or HTTPS) allowed to access the resource. 

For example, if a blob’s URL is: 

https://my_account.blob.core.windows.net/my_container/my_file

After appending the token, it looks like:

https://my_account.blob.core.windows.net/my_container/my_file?sp=r&st=TIME&OTHER_PARAMETERS

The steps below show how to use the Azure portal to generate a SAS token for a specific container. 

• Go to the Storage account page.
• In the left menu, under Data storage, select Containers. This page has a table with the list of containers under this storage account. 
• Find the row with the container for which you want to generate the token. 
• Select the three dots in the rightmost column of that row. 

• Select Generate SAS from the pop-up menu. 

In the Generate SAS window:

• Select the signing method: 
• Account key or user delegation key. 
• For this tutorial, use the account key to sign the SAS. 
• The user delegation key is based on Azure Active Directory and linked to a specific user ID via Microsoft Entra. 
• Select the type of permissions you want to authorize with this token:
• Read, write, delete, etc. 
• For this tutorial, grant just read access. 
• Enter the starting and expiration times for the validity of the token.
• Choose the IP protocols that a client can use to access your data. 
• Select the Generate SAS token and URL button. 

You can now access a private blob using the generated token and URL. When you try to access a private blob without the proper access tokens, you get an XML error that says, “The specified resource does not exist.”

Using Microsoft Entra ID for authentication

Microsoft Entra ID manages user identities and roles in one place and allows granular access to Azure resources. In addition to SAS tokens and keys, it provides another (more secure) option for managing access to Azure Blob Storage entities. You can grant specific permissions to individual users or groups by assigning them appropriate roles. This is called Role-Based Access Control (RBAC). 

RBAC facilitates fine-grained access control by having roles such as Owner, Contributor, and Reader. You can define these permissions at various levels, such as the entire Storage account or individual containers. 

For example, for a particular storage account, you can assign a user the role of Storage Blob Data Reader. That user will then be able to read only the blobs in that account but not do anything else (like write or list all the blobs). 

You can integrate Azure Blob Storage with enterprise authentication systems, using Entra IDs to authenticate users. Enterprise Azure users and applications already have (or can get) Entra IDs, making it easier to manage authentication centrally using existing enterprise credentials. 

Note that Microsoft Entra ID is the new name for Azure Active Directory (AD).

In the next section, I will show you how to assign users roles and manage access to Blob Storage entities. 

Configuring Role-Based Access Control (RBAC)

I will show you how to configure access by assigning roles for a Storage Container. The process for Storage Accounts is similar. 

• From the sidebar on the Storage container page, select Access Control (IAM). 
• On the Access Control page, select the Add button in the top bar.
• Select Add role assignment from the dropdown menu.

• On the Add role assignment page, in the Role tab, type “blob” in the search bar. This will show you the relevant preconfigured roles applicable to Blob Storage resources. 

You can choose from three types of roles for Blob Storage:

• Storage Blob Data Reader: This role has only read access to blob containers and data. 
• Storage Blob Data Contributor: This role can read, write, and delete resources. 
• Storage Blob Data Owner: Owners have full access to storage resources. 

Note that roles are inherited. For example, if you assign a user the “Owner” role for a Storage Account, that user also becomes the owner of the Containers in that account. 

• To proceed with the example, choose Storage Blob Data Contributor and select Next. 
• On the Add role assignment page, in the Members tab, click the Select members link to select the members to whom you want to assign the role you selected in the previous step. 

• In the Select members side pane, type the name of the user or application. Select the appropriate member from the search results. Click the Select button at the bottom of the side pane. 
• Select Review + assign to complete the role assignment. 

Advanced Features of Azure Blob Storage

We’ve reviewed the basics of Azure Blob Storage, but once you want to take it to the next level, here are some strategies to consider. 

Blob lifecycle management

Frequently accessed data should be stored in fast-access storage tiers like Hot. Rarely accessed data can be stored in slow but economical storage tiers, like Archive storage. 

The challenge is dynamically figuring out which blobs are no longer frequently accessed and can be moved to cheaper storage. To do this automatically, Azure comes with a Lifecycle management feature. This feature lets you dynamically move blobs to cheaper storage when they are no longer actively modified.

Lifecycle management is implemented at the Storage Account level. The steps below show how to activate and use it to add a rule to move blobs from Hot to Cool storage after 30 days of inactivity: 

• On the Storage Account page, from the left sidebar, select Data management > Lifecycle management
• The Lifecycle management page shows a list of currently active rules. 

• Select the Add a rule button from the top bar. 
• On the Add a rule page, select:
  • A name for the rule
  • The scope of the rule
  • The type and subtype of blobs to which the rule applies
• Select Next.
• The rule states that if the blob was last created or modified before N days (you need to enter the value of N), Azure will take some action (of your choosing) on it:
  • Choose the condition in the first field (block), under If. In the More than (days ago) field, enter the number of days: 30.
  • In the next field (block), under Then, choose the action - Move to cool storage.

• Add the rule.

Versioning and soft delete

In any storage system, it is essential to recover from user errors and unintended actions. Versioning and soft delete are the two ways Azure Blob Storage uses to help you undo mistakes. 

Versioning 

It is useful to be able to restore the previous versions of a file if you accidentally save the wrong changes. Blob versioning allows you to automatically retain older versions when you modify a blob. To enable versioning:

• On the Storage account page, select Data management > Data protection from the sidebar. 
• On the Data protection page, under Tracking, check the box for Enable versioning for blobs
• Select whether you want to:
  • Retain all previous versions.
  • Delete older versions after a few days. 

• Save the changes. 

Versioning is now enabled on your storage account!

Soft-delete 

We have all accidentally deleted the wrong data. It is useful to be able to recover the lost data when this happens. 

Blob soft delete is a feature that retains the deleted data for a specified number of days. During this retention period, you can restore the soft-deleted entity. After the retention period, the data is permanently deleted. This protects data from accidental deletion and overwriting.

To activate and use soft-delete: 

• On the Storage account page, select Data management > Data protection from the sidebar. 
• On the Data protection page, select the checkboxes for:
  • Enable soft delete for blobs
  • Enable soft delete for containers
• Enter the number of days for which to retain deleted data.
• Save the changes. 

You have now activated soft-delete on your storage account!

Data encryption and secure access

By default, Azure employs AES-256 symmetric encryption at rest to keep data secure. An attacker with physical access to the hard drive on which the data is stored cannot decrypt the data without the encryption keys. Many organizations need this feature for security and compliance with privacy regulations. 

By default, the user does not need to handle the additional workload of maintaining encryption keys. Azure takes care of it under the hood. However, you can choose to use the key vault and provide your keys for added security:

• On the Azure portal, type “key vault” in the search bar at the top.
• Select Key vaults.

• The Key vaults page shows the list of key vaults you have created. 
• Select the Create button in the top bar. 
• On the Create a key vault page:
  • Select the resource group for which you want the keys in this vault to be used.
  • Enter the name of the vault and the region where you want it stored.
• Select Review + Create.
• Check the details and select Create.

Creating the new key vault takes a while and then directs you to its page. 

After creating the key vault, you can now add your keys and secrets:

• From the sidebar, go to Objects > Keys.
• Select the Generate/Import button from the top bar. 
• Select whether you want to generate or import a new key. You can generate your key locally and upload it to Azure or generate the key on the Azure platform. 
• Enter the name of the new key.
• Select the appropriate details of the key (type, size, algorithm, etc.). 
• Optionally, you can also set the activation and expiration dates of keys. 

• Select Create. 

You have now added a key to your keyvault. 

Geo-replication for disaster recovery

The safer you need your data to be (from technical or natural calamities), the more redundancy you build into your storage system. Redundancy involves replicating the same data across multiple devices and/or locations and comes at a cost. You can choose from various levels of redundancy: 

• Locally redundant storage (LRS): Data is replicated within the same data center. This is the cheapest option. 
• Zone-redundant storage (ZRS): This creates three or more replicas across different data centers in your geographical region. 
• Geo-redundant storage (GRS): Data centers across geographical regions keep replicas of your storage account. This is the safest option. 

Cost Management and Optimization in Azure Blob Storage

Cloud storage, for all its benefits, can become expensive if costs are not managed properly. In this section, I will give an overview of how to optimize and manage costs for Azure Blob Storage. 

Understanding Azure Blob Storage pricing

Azure Storage has various tiers of pricing depending on the options and additional services offered:

• Frequency of access: Frequently accessed data needs to be stored on devices with faster read and write speeds. Conversely, rarely accessed data can be stored on slower and cheaper devices. 
• Hot: This is typically built on modern SSDs, which are fast but expensive. They give the best user experience. 
• Cool: This is suitable for infrequently accessed data like backups. It is also sometimes used for larger files like videos, which can be expensive to store on hot storage. 
• Archive: Data that needs to be accessed only rarely is archived. Conventionally, archives are stored on tapes. So, storage costs are low, but it is expensive to retrieve data (due to sequential access). 
• Redundancy: The safer you need your data to be (from technical or natural calamities), the more redundancy you build into your storage system. Redundancy involves replicating the same data across multiple devices and/or locations and comes at a cost.
• Locally redundant storage (LRS): Data is replicated within the same data center. This is the cheapest option. 
• Zone-redundant storage (ZRS): This creates three or more replicas across different data centers in your geographical region. 
• Geo-redundant storage (GRS): Data centers across geographical regions keep replicas of your storage account. This is the safest option. 
• Data transfer: 
• Typically, you can transfer data within the same region across Azure services for free. 
• Data ingress (transferring data into Azure Storage) is also typically free. 
• Data egress (transferring out of Azure), such as internet users accessing your data,  incurs costs. The cost of egress is typically volume-dependent. 

Cost optimization tips

Based on the discussion above, I recommend a few best practices to prevent cloud storage costs from spiraling out of control. 

• Take advantage of the different storage types discussed above. Move infrequently accessed data to cheaper storage tiers, like Cool or Archive. 

• A caching service like a CDN can also help improve latency. This will allow you to store large files, like videos, in Cool storage. 
• Enable and set appropriate lifecycle policies to move unused blobs to lower-cost storage tiers. 

Troubleshooting Azure Blob Storage

Although Azure Storage is a widely used product, errors sometimes occur. 

Common issues with blob uploads

Network issues and permission errors are among the most common types of problems. Slow connections and misconfigured proxies frequently lead to network errors, which can be best resolved by checking your connection and configuration. 

It is also common to get errors when trying to upload large files. A momentary loss in network connectivity can lead to the upload being stalled. To address such issues, check Azure’s retry policies and configure them as needed. Another pragmatic solution is to break up large uploads into smaller chunks. Upload each chunk separately using the Put Block operation. 

Permission errors are caused when your access keys or tokens, or Entra ID, are not authorized to perform your desired operation (e.g., read or write data) to the storage container or account. It is possible your keys have been refreshed, or your token has expired. The container manager can help you get the correct access rights. 

Monitoring blob storage usage

Azure includes various tools to monitor the performance of Azure Storage. We will cover two of the most common choices: 

• Metrics under Azure Storage 
• Azure Monitor

Azure Storage Metrics

Azure Storage comes with an integrated monitoring tool. To access it: 

• From the Storage account page, select Monitoring > Metrics
• The Metrics page shows a graph, which is initially empty. You need to choose what information to display in this graph. 

• In the dropdown menu bar (under Scope) at the top of the chart, select the storage entity you want to monitor:
  • Under Metric Namespace, choose either (storage) Account or Blob.
  • Under Metric, choose a relevant metric, such as Blob count, Ingress, capacity, Transactions, etc.
  • At the top right, customize the time frame over which you want to monitor the metric.
  • The resulting chart shows the performance of the storage entity using the chosen metric over the chosen period.

Azure Monitor 

Azure Monitor is a high-level tool that you can use to monitor various Azure resources. To access Azure Monitor:

• On the Azure portal, type monitor in the search bar at the top. 
• Select Monitor from the search results.

• On the Monitor homepage, from the sidebar, select Insights > Storage accounts. 

• On the Monitor | Storage accounts page, select the storage account for which you want to see the insights. 
• The Insights page shows various useful metrics, such as transactions by storage type, latency, availability, and more. 

• At the top of the page, you can customize the time range for which to show the metrics. 
• Use the Customize button in the top bar to modify the layout of various tables on the page. 

Conclusion

Cloud storage is an integral part of modern software. In this article, we discussed how to get started with Azure Blob Storage and how to use it to upload and download blobs. We also covered many practical aspects of Blob Storage, such as managing access rights and monitoring usage to control costs. 

Microsoft’s Azure and Amazon’s AWS offer many overlapping services. To decide which better suits your individual needs, go through our AWS vs. Azure comparison. Finally, if you have an upcoming interview involving Azure services, we have a set of relevant interview questions for you!