The connection resiliency feature on EF6 automatically deals with database operations that can fail due to transient conditions and need to be retried one or more times until they succeed. The mechanism relies on the ability to observe certain kinds of failures as exceptions on the client side and on the assumption that when one of those failures is observed it will always be caused by an operation that can be retried, i.e. that the operation is either idempotent (a rare occurrence in data modification operations in EF) or that its failure guarantees no durable side effects (in fact, EF was updated to ensure that by default all database interactions that can modify the data stored on the database are wrapped in a transaction, helping attain this no durable effects on failure).
We have identified a situation in which those assumptions are insufficient: due to unstable connectivity it is possible for transaction success notifications to not be properly or timely received on the client side, which can result in an exception being observed on the client that does not correspond to a re-triable operation failing on the server.
Incidentally, if we were to retry on this kind of situation it would cause data corruption, since generally speaking data modification operations produced by EF are not idempotent.
On the other hand even if we don’t retry, in the case of SaveChanges no store generated values are propagated into the objects if a failure happens and therefore we are left with change tracking state that is inconsistent with the database. The application has to then handle the situation by throwing away the context.
What can be done?
1. Our highest priority for EF6 should be to avoid causing data corruption. For that we need to make sure that when such situation arises it is not presented to us as one of the transient failures we should retry
2. Long term we should look into handling this situation gracefully as an extension of connection resiliency work:
a. Add a ‘transaction log’ entity and table to which on each transaction we will insert a new row with a client generated GUID
b. Any time we recognize a failure on a transaction commit we can re-query the database for the GUID value we just generated. If the row is there then we know the transaction didn’t fail and if we are in SaveChanges() we can resume SaveChanges() as normal (including back propagation of store generated values).
3. Longer term we should keep working with the Managed Providers team to track any work they can do to fix this on their end in a way that doesn't cause as much overhead
4. Possible guidance for EF5 and EF6: part of the solution described in #2 can be implemented as a workaround today: user can add a ‘transaction log’ table, insert a row on with a client generated GUID on each operation and know that the transaction succeeded even if a failure was detected if the GUID gets stored. The part that cannot be implemented today is to resume the rest of SaveChanges: while it is possible to ‘accept all changes’ on the context it is not possible to propagate store generated values from the database into the objects in memory if SaveChanges fails (note that if there are not store generated values in the model at all this shouldn’t be a problem).
5. Another alternative form of the workaround is to issue an UPDATE on a table with concurrency control on a timestamp on every transaction. Doing this will cause any subsequent retry to fail with a concurrency violation. Note that this workaround won’t recover store generated values.
The following blog post describes a workaround similar to #4 implemented for applications using ADO.NET with stored procedures:
[SQL Database Connectivity and the Idempotency Issue](http://blogs.msdn.com/b/adonet/archive/2013/03/11/sql-database-connectivity-and-the-idempotency-issue.aspx)
We have identified a situation in which those assumptions are insufficient: due to unstable connectivity it is possible for transaction success notifications to not be properly or timely received on the client side, which can result in an exception being observed on the client that does not correspond to a re-triable operation failing on the server.
Incidentally, if we were to retry on this kind of situation it would cause data corruption, since generally speaking data modification operations produced by EF are not idempotent.
On the other hand even if we don’t retry, in the case of SaveChanges no store generated values are propagated into the objects if a failure happens and therefore we are left with change tracking state that is inconsistent with the database. The application has to then handle the situation by throwing away the context.
What can be done?
1. Our highest priority for EF6 should be to avoid causing data corruption. For that we need to make sure that when such situation arises it is not presented to us as one of the transient failures we should retry
2. Long term we should look into handling this situation gracefully as an extension of connection resiliency work:
a. Add a ‘transaction log’ entity and table to which on each transaction we will insert a new row with a client generated GUID
b. Any time we recognize a failure on a transaction commit we can re-query the database for the GUID value we just generated. If the row is there then we know the transaction didn’t fail and if we are in SaveChanges() we can resume SaveChanges() as normal (including back propagation of store generated values).
3. Longer term we should keep working with the Managed Providers team to track any work they can do to fix this on their end in a way that doesn't cause as much overhead
4. Possible guidance for EF5 and EF6: part of the solution described in #2 can be implemented as a workaround today: user can add a ‘transaction log’ table, insert a row on with a client generated GUID on each operation and know that the transaction succeeded even if a failure was detected if the GUID gets stored. The part that cannot be implemented today is to resume the rest of SaveChanges: while it is possible to ‘accept all changes’ on the context it is not possible to propagate store generated values from the database into the objects in memory if SaveChanges fails (note that if there are not store generated values in the model at all this shouldn’t be a problem).
5. Another alternative form of the workaround is to issue an UPDATE on a table with concurrency control on a timestamp on every transaction. Doing this will cause any subsequent retry to fail with a concurrency violation. Note that this workaround won’t recover store generated values.
The following blog post describes a workaround similar to #4 implemented for applications using ADO.NET with stored procedures:
[SQL Database Connectivity and the Idempotency Issue](http://blogs.msdn.com/b/adonet/archive/2013/03/11/sql-database-connectivity-and-the-idempotency-issue.aspx)