error found in 8.1.0.148 when reading in scripts

yzguyyzguy Posts: 23 Bronze 2
RedGate Compare Engine info:
Version = 8.1.0.148
Runtime Version = v2.0.50727
(I am using the newer one, because I have to read Redgate 8.1 snapshots)

I would attach a zip file with the view script and DB info xml file I am using, but I don't think I can attach files. The text of the view is (I think it has something to do with the comments):
SET QUOTED_IDENTIFIER Off
GO
SET ANSI_NULLS ON
GO

CREATE VIEW v_focus_lists
AS
select * from symbol b
where 1=1 -- list_type_code = 'F' and end_date is null LG: 10/23/06
/*comment*/

GO



I am using this code (I get the error on the db.Register line). If I try to compare a scripts location with this view in it, I get an error.

Database db = new Database();
Options _projectOptions = Options.Default;

db.Register(@C:\DBScripts, null, _projectOptions);

This is the stack trace:


at RedGate.SQLCompare.ASTParser.StructureLexer.nextToken()
at antlr.TokenBuffer.fill(Int32 amount)
at antlr.TokenBuffer.LA(Int32 i)
at antlr.LLkParser.LA(Int32 i)
at RedGate.SQLCompare.ASTParser.StructureParser.bracketedSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.collateSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.unarySubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.bitwiseSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.divideOrMultiplySubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.addOrSubtractSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.expressionSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.expression()
at RedGate.SQLCompare.ASTParser.StructureParser.comparisonSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.isInSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.notSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.andSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.orSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.searchSubclause()
at RedGate.SQLCompare.ASTParser.StructureParser.searchCondition()
at RedGate.SQLCompare.ASTParser.StructureParser.wherePart()
at RedGate.SQLCompare.ASTParser.StructureParser.simpleSelectExpression()
at RedGate.SQLCompare.ASTParser.StructureParser.subSelectExpression()
at RedGate.SQLCompare.ASTParser.StructureParser.selectExpression()
at RedGate.SQLCompare.ASTParser.StructureParser.selectStatement()
at RedGate.SQLCompare.ASTParser.StructureParser.createViewStatement()
at RedGate.SQLCompare.ASTParser.StructureParser.createStatement()
at RedGate.SQLCompare.ASTParser.StructureParser.statementFirstInBatch()
at RedGate.SQLCompare.ASTParser.StructureParser.statementBatch()
at RedGate.SQLCompare.ASTParser.StructureParser.expr()
at RedGate.SQLCompare.Engine.DatabaseFromScriptObjectResolver.ColumnsInObject(ISqlObject obj)
at RedGate.SQLCompare.Engine.DatabaseFromScriptObjectResolver.a(SchemaObject )
at cY.a()
at RedGate.SQLCompare.Engine.Database.Register(String path, ScriptDatabaseInformation dbinfo, Options options)[/quote]

Comments

  • Brian DonahueBrian Donahue Posts: 6,590 New member
    Hi Mark,

    Thanks for your post. Can you please let us know the exception type, eg do you get a "NullReferenceException"? Off the top of my head, I would suggest that there is a problem in the script containing the "symbol" object that is preventing it from being registered.

    You could try running the SQL Compare UI and checking for warnings when trying to compare the scripts folder. You could access the list of warnings programatically, but this may take a bit more time.
  • yzguyyzguy Posts: 23 Bronze 2
    The error is :
    antlr.TokenStreamRecognitionException was unhandled
    Message="unexpected char: 0xFFFF"
    Source="RedGate.SQLCompare.ASTParser"

    It has nothing to do with the objects that are actually in the script, the error is in the reading in of the script. This is not my actual view, I just deleted all of the non essential stuff so that the error was easily reproduceable.

    I have removed even more from the script and am still able to reproduce it with this script:
    SET QUOTED_IDENTIFIER Off
    GO
    SET ANSI_NULLS ON
    GO

    CREATE VIEW v_focus_lists AS
    select * from s -- list_type_code = 'F' and end_date is null LG: 10/23/06
    /**/

    GO

    Except this time I went further to determine that if I remove the /**/ line, or if I remove one of the spaces between the 'F' and the 'and' (there are 2 spaces there, which don't show up in message boards), or if I remove the date in the comment, it does not error. There is some cobination here that is causing the parser to throw an error (unexpected char: 0xFFFF)


    I can not easilly run the sql compare UI on this, as comparing to a script folder is only an option in the pro version, and I do not have the pro version of SQL compare. I expect this to be a bug in the parser, and if you look at the stack trace, you can see which function is actually generating the error.
  • Brian DonahueBrian Donahue Posts: 6,590 New member
    There must be something about the file, for instance, corruption in the raw bytes that suggest a Unicode continuation followed by invalid data.

    Can you copy and paste the script into a brand-new file and see if you have the same problem?
  • yzguyyzguy Posts: 23 Bronze 2
    I have tried that (using notepad), and get the same result. It is something about this string of chars.

    if you want I can email you the file. I'd post it, but I don't think this board allows attachments.
  • yzguyyzguy Posts: 23 Bronze 2
    Ok I can give exact steps to reproduce.

    Using RegGate.SqlCompare.Engine
    Version 8.1.0.148
    Runtuime Version v2.0.50727
    (downloaded from this forum)

    Copy this sql:
    SET QUOTED_IDENTIFIER Off
    GO
    SET ANSI_NULLS ON
    GO

    CREATE VIEW v_focus_lists AS
    select * from s -- list_type_code = 'F' and end_date is null LG: 10/23/06
    /**/

    GO


    Create these directories:
    C:\DBScripts\Views

    Create a new text file in the Views dir, and name it "dbo.v_focus_lists.sql"

    Open in notepad and paste in code from above.
    Add an extra space (so 2 total) between the "'F'" and the "and"
    Add an extra space (2 total) between the "s" and the "--"

    Save the file

    Then run this code:
    using RedGate.SQLCompare.Engine;
    using RedGate.SQLCompare.Engine.ReadFromFolder;


    private void test()
    {

    Database db = new Database();
    string scriptLocation = "c:\\DBScripts\\";

    ScriptDatabaseInformation info = new ScriptDatabaseInformation();
    FolderDataSource folder;
    folder = new FolderDataSource(scriptLocation);
    // Read the schema from scripts folder
    db.Register(scriptLocation, folder.DatabaseInformation, Options.Default);



    }
  • Brian DonahueBrian Donahue Posts: 6,590 New member
    Can you try getting SQL Compare from the latest cumulative patch and referencing your libraries from there? This works for me. RedGate.SQLCompare.Engine.dll version 8.1.2.29.

    http://www.red-gate.com/messageboard/vi ... php?t=9860
  • yzguyyzguy Posts: 23 Bronze 2
    that link updated my RedGate.SQLCompare.Engine.dll to:
    Version 8.1.2.37

    and is still doing the same thing.

    I don't know if it matters but the other redgate dll's I have included are:

    RedGate.Shared.SQL version 8.0.0.21
    RedGate.Shared.Utils version 8.0.0.21
  • Brian DonahueBrian Donahue Posts: 6,590 New member
    I'm sorry, but your reproduction doesn't work for me. The database registers fine. What am I missing?
  • yzguyyzguy Posts: 23 Bronze 2
    I started from scratch.

    New project using VS Team System 2008 Database Edition Version 9.0.30729.1
    Visual C#
    Windows
    Windows Forms Application


    Add reference to:
    C:\Program Files\Red Gate\SQL Comparison SDK 8\Assemblies\SQL Compare\RedGate.Shared.SQL.dll (8.0.0.21)
    C:\Program Files\Red Gate\SQL Comparison SDK 8\Assemblies\SQL Compare\RedGate.Shared.Utils.dll (8.0.0.21)
    C:\Program Files\Red Gate\SQL Compare 8\RedGate.SQLCompare.Engine.dll (8.1.2.37)

    I noticed that I have this copy of the same dll but this one is an older version: (I'm not using this one, I just noticed it)
    C:\Program Files\Red Gate\SQL Comparison SDK 8\Assemblies\SQL Compare\RedGate.SQLCompare.Engine.dll


    In form1 add using statements
    using RedGate.SQLCompare.Engine;
    using RedGate.SQLCompare.Engine.ReadFromFolder;


    In form load put this code:
    Database db = new Database();
    string scriptLocation = "c:\\DBScripts\\";

    ScriptDatabaseInformation info = new ScriptDatabaseInformation();
    FolderDataSource folder;
    folder = new FolderDataSource(scriptLocation);
    // Read the schema from scripts folder
    db.Register(scriptLocation, folder.DatabaseInformation, Options.Default);


    Copy this text:
    SET QUOTED_IDENTIFIER Off
    GO
    SET ANSI_NULLS ON
    GO

    CREATE VIEW v_focus_lists
    AS
    select * from symbol b
    where 1=1 -- list_type_code = 'F' and end_date is null LG: 10/23/06
    /*comment*/

    GO


    Create the directory c:\DBScripts\views then create a new text file (I used notepad) named dbo.v_focus_lists.sql in the views dir and in it paste the above text, then add an extra space between the "'F'" and the "and" and between the "s" and the "--"

    Save the file, and run the app.

    I still get this error on the db.Register line:

    "unexpected char: 0xFFFF"


    I also moved these files locally to the project, changed the reference to the local copy, and zipped up the project and the script so that it can just be unzipped and run (assuming the pc has a license for the redgate products). Just unzip it to your C drive and it will create a RedgateTestApp dir with the script, dlls and the .net project in it.

    I don't have a way to post an attachment here, and I don't have an easy way to post something publicly available from work, so when I get home tonight I'll upload the zip and post a link to it here.

    edit-
    here is a link to the zip file containing the project and the script:
    http://www.1bad69.com/redgatetestapp.zip
  • Brian DonahueBrian Donahue Posts: 6,590 New member
    Thanks for the files. I see that there is a problem, but I'm still looking for the cause. I think you have caused some odd condition where the SQL Parser thinks there is a character not valid for a UTF-8 file. This is a weird one!
  • yzguyyzguy Posts: 23 Bronze 2
    yeah, I'm pretty sure it is a specific string of chars that is throwing the parser off. I know I can fix this one example, but I don't know the exact pattern that is causing this (in order to look for it in the future), and the view that we happened to find this on, is code from a vendor that we really don't update.
  • Brian DonahueBrian Donahue Posts: 6,590 New member
    Hi,

    I think I am coming to the conclusion that I can't work out what the problem is and I'll have to get another pair of eyes on it.
    • delete space from the comment and the comment identifier (-- and l), the database registers.
    • put the space back in, sitll registers. It definitely should not, considering I put the file back the way it was!

    I'd still say that there must be some rubbish in the file at this point, but probably need an expert at text file formats to help work it out.
  • yzguyyzguy Posts: 23 Bronze 2
    I don't think the file is corrupted, as I have repeated this many times, using a whole new file and pasting in text to notepad.

    that is surprising that you can remove, then put back the space, and see it register correctly. I have done that, and mine goes back to not registering.
  • I'm pleased to inform you that we now have a patch version of SQL Compare that fixes this issue.

    If you would like to try the patch, you can download it from here

    Let me know if it fixes the issue or not.
    Chris
  • yzguyyzguy Posts: 23 Bronze 2
    I had to add references to
    RedGate.SQLCompare.Rewriter.dll
    and
    RedGate.SQLCompare.ASTParser.dll

    but, now it works! :)

    Thanks,
    Mark
Sign In or Register to comment.