How to “MAX” out your Fast Refresh Materialized Views

By | In Oracle | December 10th, 2010

With a very simple materialized view that contains an aggregate function like MAX, Oracle can perform a fast refresh of the materialized view. However, once we start adding some complexity to the materialized view query, Oracle starts to either error out on the fast refresh, or will disallow the creation of the fast refresh materialized view in the first place. At Blue Gecko, we occasionally come across a problem, while providing remote DBA services for clients, that cannot be solved by only reading the Oracle documentation. One such problem was for a client who wanted to use a MAX aggregate function in a complex, fast refresh materialized view. The Oracle documentation says that this is not possible. As a materialized view query goes from simple to complex, Oracle’s fast refresh mechanism starts to fail. Is it possible to overcome these limitations with a fast refresh materialized view? And if so, how? Can we have our MAX and FAST REFRESH it too? The answer is yes and I’ll show you how.

How does a simple materialized view query with a MAX aggregate function work?

Before we dive into how to make a complex materialized view (mview) query work, let’s look at a simple mview query. How exactly does a fast refresh of a simple mview query work when it includes an aggregate function like MAX? Let’s work through an example. To show how oracle performs a fast refresh of a simple materialized view with a MAX aggregate function, we will start with a fairly generic “orders” table and create a materialized view that contains the maximum order amount for each customer.

Let’s look at the contents of the orders table and the contents of the mview based on some data I added to the orders table:

Let’s now update a row to change the max_order_amount value for one of the customers. We will then refresh the mview and see the change.

We can see that customer_id of 1020 has a new max_order_amount of 100.11, up from 99.15. That went well. Let’s update the data back to their original values to verify that fast refresh works when the max_order_amount decreases as well.

Pretty cool. So how does Oracle accomplish this? If we take a 10046 trace of the fast refresh command we see that Oracle issues a DELETE against the mview for the effected rows in the materialized view log (mlog) table. In this case, it is only for the one customer_id that we updated. It then turns around does an INSERT of those rows again, getting the data from a SELECT against the orders table and filtering down to just the customer_id values it wants by using the mlog table. The mlog table in this case is called mlog$_orders and was created in our original CREATE MATERIALIZED VIEW LOG command. Here are the DELETE and INSERT statements from the trace:

We can see that these DELETE and INSERT statements are fairly complicated for even our simple mview query. As our mview query goes from simple to complex, the logic to generate the appropriate DELETE and INSERT statements becomes more and more difficult.

What happens when we make the mview query more complex?

The example above shows that a simple mview with an aggregate function will fast refresh, but what happens when the mview query becomes more complex? As we will see, the fast refresh ability starts to fail. One change we could make that will cause Oracle to stop being able to fast refresh the materialized view is to add a WHERE clause to our mview query. There are other things we could change to make it complex as well. For example, we could add a SUM aggregate function, or a COUNT aggregate that includes a CASE statement. We won’t go into all the different ways an mview query could become complex here. Lets just look at what happens when we add the WHERE clause and try to refresh the mview. For our WHERE clause, let’s restrict our query to only those orders that are in a “closed” state. We will need to add the state column to the list of filter columns on the materialized view log. Let’s do that and also change the name of the mview from “orders_by_cust_mv” to “closed_orders_by_cust_mv” while we are at it.

So far so good, but we haven’t updated any data yet. Let’s see what happens when we do.

And we blow up. We get an ORA-32314 error and our mview will no longer fast refresh. We could try to use the DBMS_ADVISOR.TUNE_MVIEW procedure to try to see how Oracle would recommend modifing this mview in order to make it fast refresh, but we strike out there as well:

So how can we fast refresh a complex mview with a MAX function?

Many people would stop here. They would run some Internet searches, check their favorite Oracle blogs and come up empty on how to proceed. Finding nothing, they would go back to their customers and give them the unpopular choice of either removing the aggregate functions from their mview or switching the mview to be a complete refresh. Is possible to work around this apparent roadblock? Yes, but not solely within the functionality of mviews that Oracle provides. To make this work we are going to need to utilize mviews in conjunction with an additional table we create and some PL/SQL. Here is how. In the simple mview example, Oracle was deleting and re-inserting the MAX value based on the changes that it saw in the mlog table. We’ll do the same thing, although I’m going to use the MERGE SQL statement to replace the DELETE and INSERT (more on that later). After we have the aggregate data we need in this new table, we will create a materialized view on top of both the original orders table and this new aggregate table. We will tie it all together with a PL/SQL procedure that updates the aggregate table and refreshes the mview at the same time. First let’s create the new aggregate table and populate it with all the data:

In order to get our mview to fast refresh we will need a materialized view log on the new aggregate table as well, so let’s do that first. We then modify the original mview query to pull the max_order_amount data from this new aggregate table. We join back to the orders table on the customer_id column to complete the new version of the materialized view.

The initial data looks good. The max_order_amount column does not get updated when we refresh the mview, and it shouldn’t at this point. The aggregate table is a plain old table, separate from the orders table. There is nothing currently keeping the aggregate table in sync with the orders table. Let’s run a test to show that the aggregate table does not get updated when we update the orders table. When we then refresh the mview, we should expect the data to be out of sync. We’ll fix this in a minute.

We updated the orders table, but the mview does not see the change as we expected. In order to have the data in the aggregate table updated properly, we’ll want to run another MERGE query each time we do a fast refresh. This merge query will be slightly different than the one we created to initially populate the aggregate table. We’ll do what Oracle did and use the materialized view log table to filter down the number of rows we want to update. Let’s take a look back at the original MERGE statement we used to populate the aggregate table:

We will want to update the aggregate data with every fast refresh, so let’s create a PL/SQL procedure that will combine this MERGE query with the refresh of the mview. We’ll grab the customer_id values that changed from the mlog table and update all the rows in the aggregate table that match those customer_id values. If the max_order_amount doesn’t change, we don’t need to update the aggregate table. For this optimization, we will add a WHERE clause to our UPDATE. I’ll underline the changes to the MERGE statement in our new refresh procedure below:

Now if we reset the order data back to the way it was and try our update again, we should see the change reflected in the mview.

It works just the way we wanted it to, but we’re not quite done yet.

Potential issues with this approach

One potential complication we might encounter is if someone manually refreshes the mview without going through the PL/SQL procedure we created to do the refresh. If this happens, then the rows in the materialized view log table will be deleted without the aggregate table being updated. That would be bad as the aggregate table could get out of sync with the orders table and never get back in sync. We can prevent this by creating a second mview on the orders table that does nothing except pin the mview log entries to the table until we run the merge into the aggregate table. Let’s call this a “ghost” mview. This ghost mview is an added layer of redundancy. If you feel like you have control over how and who may issue refresh commands to your mviews, this second mview is not strictly necessary. For safety, I recommend creating this additional mview to pin the mlog entries until we run our MERGE command. Let’s look at what can happen without the additional ghost mview to pin the mlog entries.

We can see that the accidental refresh of the closed_orders_by_cust_mv materialized view removes the entries from the mlog. The subsequent execution of the PL/SQL refresh procedure we created misses the update because these mlog rows no longer exist. Our data is out of sync and nobody is happy. Let’s add the ghost mview to pin the mlog entries. Then we will modify the procedure to refresh the ghost mview after the merge and try this experiment again.

When we run our update statement now, we see that an accidental refresh of the mview does not remove the entries from the mlog table.

That looks better. The ghost mview to pin the mlog entries does the trick. The accidental refresh of the closed_orders_by_cust_mv materialized view does not remove the mlog rows and the subsequent procedure call correctly updates the aggregate table.

Why use MERGE instead of DELETE and INSERT

I mentioned earlier that I was going to use a MERGE SQL statement instead of a DELETE followed by an INSERT. Yet we know that Oracle does a DELETE followed by an INSERT when they update their aggregate data in the simple mview query case. Why don’t I follow that logic? I’m using MERGE for performance. If the max_order_amount doesn’t actually change based on any updates to the orders table, I would rather not DELETE and INSERT rows in the aggregate table. This would be unnecessary and would create rows in the aggregate table’s materialized view log, in turn causing us to do more work than we have to during the fast refresh of the mview. Will my data be correct? The correctness of the data remains because I’m joining the aggregate table back to the orders table, and I’m still filtering rows in the mview itself. You’ll see that with the MERGE SQL statement, I do leave some bogus data in the aggregate table when I either remove or modify data such that it no longer matches the WHERE clause. A DELETE and INSERT would remove these bogus rows. However, because of the join in the mview, this is not an issue. Let’s see what happens when I update all the orders to the “open” state.

The data in the aggregate table is wrong, but the final mview is remains correct. If I wanted to expose this aggregate table to the end users, then I would change my MERGE to the more expensive DELETE and INSERT. And there you have it. A workaround for using aggregate functions like MAX in a complex, fast refresh materialized view.

A final look at the code

This code was tested on both 10g and 11g versions of Oracle. We started with the following materialized view definition that would not fast refresh after updates to the data.

And we ended with a reworked materialized view, an additional table and PL/SQL procedure to support the MAX aggregate data, and a ghost mview to pin the mlog entries for safety.

Dallas Willett Blue Gecko – Remote DBA Services

Contact Us

Leave a Reply

Your email address will not be published.
Required fields are marked (*).