Output char buffer is too small

Seany

Hi all,

I am getting the following error when I try to generate a data compare script from one DB to another.

The message is:

The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars

Thanks in advance

James B

Thanks for your post.

I've looked through our historic data here, and this error message has only cropped up once before. Unfortunately, the customer on that occasion never responded to our request for more detail so we never discovered it.

Thinking at the time was that is potentially a problem in the .net framework, but could you give a little more information:

- What exact version of SQL Data Compare are you using?
- Does this occur when working with two actual databases, or are you using script folders?
- Does the process succeed if you let SQL Data Compare perform the update (as opposed to generating a script)?
- Do your database(s) use any unusual collation?
- What do you have set in Tools > Application Options under the Synchronization Scripts Encoding option? Do any other options here work?

If you want to contact us directly to work through this, please mail us directly at support@red-gate.com, quoting F0045000 in the subject line.

Seany

Thanks for the reply James.

To answer some of your queries:

v8.0.2.5 Both databases are in the same instance. Yes, SQL Compare does synchronise the DB's as opposed to when it tries to generate the script(s). It is a client's DB that has been live since ~SQL2000 and is currently running in a SQL 2008 environment. There are no unusual collations. There are ~1.5million rows. Current encoding is set to UTF16. I will try some of the others in the mean time.

James B

OK - it may also be worth turning up the logging to "verbose" to see if it gives anything more specific.

To do this, right-click the main SQL Data Compare window titlebar, and select Min Logging Level > Verbose.
Restart the app and generate your error, then check the log by right clicking once more on the titlebar and picking "Locate Log Files", then opening the most recent one.

James B

Also - are you mapping together any differing datatypes that may be incompatible?

Bastiaan Molsbeck

Hi Seany,

I'm experiencing the same error.
I'm using version 8.0.2.1 of the RedGate.SQLDataCompare.Engine.dll.

Did you ever solved your problem?

Kind regards,
Bastiaan.

Brian Donahue

Hi Bastiaan,
If you are using the SDK, can you get a stack trace from the debugger when this error occurs? It's difficult to isolate the code that throws this without some direction.

Bastiaan Molsbeck

Yes, the full stack trace is:

System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
at System.Text.UnicodeEncoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(FieldType , BinaryReader , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.b(FieldType , BinaryReader , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(Object[] , RowType& , BinaryReader , FieldPairs , List`1[] , Boolean , Byte[]& )
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.a(BinaryReader , Int32 , Int32 , Int64 , FieldPairs , List`1[] , Boolean )
at RedGate.SQLDataCompare.Engine.ResultsStore.StoreEnumerator.MoveNext()
at bC.a(aF , TableDifference , Reader , RowType , Boolean , ProgressPercentageTracker , SelectionDelegate , Boolean )
at RedGate.SQLDataCompare.Engine.SqlProvider.b(ComparisonSession , SelectionDelegate , Boolean )
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, Boolean runOnTwo)

Brian Donahue

I'm not an expert on text encoding but I think there may be corrupt data in the field...

Bastiaan Molsbeck

No, the field does not contain corrupt data.
Do you have any other suggestions?

Edit: when using version 9.0 of the SQL Data Compare Engine the same error occurs, but with a slightly different stacktrace:

System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding 'Unicode' fallback 'System.Text.DecoderReplacementFallback'.
Parameter name: chars
at System.Text.Encoding.ThrowCharsOverflow()
at System.Text.Encoding.ThrowCharsOverflow(DecoderNLS decoder, Boolean nothingDecoded)
at System.Text.UnicodeEncoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, DecoderNLS baseDecoder)
at System.Text.DecoderNLS.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex, Boolean flush)
at System.Text.DecoderNLS.GetChars(Byte[] bytes, Int32 byteIndex, Int32 byteCount, Char[] chars, Int32 charIndex)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#tN(FieldType type, BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#ceL(FieldType fieldType, BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#beL(BinaryReader reader, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.#deL(Object[] values, RowType& rowType, BinaryReader reader, FieldPairs fieldPairs, List`1[] types, Boolean summary, Byte[]& rowDifferences)
at RedGate.SQLDataCompare.Engine.ResultsStore.Reader.GetRow(BinaryReader reader, Int32 valuesCount, Int32 currentIndex, Int64 currentOffset, FieldPairs fields, List`1[] types, Boolean summary)
at RedGate.SQLDataCompare.Engine.ResultsStore.StoreEnumerator.MoveNext()
at #8ZK.#n2K.#QeL(#x1K writer, TableDifference summary, Reader rs, RowType resultsStoreType, Boolean runOnTwo, ProgressPercentageTracker progress, SelectionDelegate select, Boolean showComments)
at RedGate.SQLDataCompare.Engine.SqlProvider.#DbL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, SelectionDelegate select, Boolean runOnTwo)
at RedGate.SQLDataCompare.Engine.SqlProvider.GetMigrationSQL(ComparisonSession session, Boolean runOnTwo)

Bastiaan Molsbeck

But when using version 10.0 of the SQL Data Compare Engine, the error does NOT occur! The migration SQL script is generated successfully.

Problem solved!