Showing posts with label fields. Show all posts
Showing posts with label fields. Show all posts

Tuesday, March 20, 2012

3 quick queries

I want help with a couple of SQL queries.
I have two Tables Table A and Table B.
Both tables have the same two fields Name and Hobbies.
One Name can appear beside multiple hobbies in each table.

There are three queries I wish to run.
1) Find all the Hobbie and Name combinations in Table B not in Table A
only for Names that exist in Table A
2) Find all Hobbies and Name combinations in Table A not in Table B
3) Return all data in Table B that contains a Name that exists in Table
A

Regards,
CiarnLooks like homework to me. Have you looked up some example queries and
tried to adapt them to your application? What have you tried so far?

--
David Portas
SQL Server MVP
--|||No its not homework, what I need to do is slightly different, but
thought if I got the answers to these three I'd be able to figure it
out.
I've been playing with inner and outer joins, but can't figure out what
I need to do.

Regards,
Ciarn

BTW what does MVP stand for?|||On 4 Apr 2005 08:48:30 -0700, chudson007@.hotmail.com wrote:

>I want help with a couple of SQL queries.
>I have two Tables Table A and Table B.
>Both tables have the same two fields Name and Hobbies.
>One Name can appear beside multiple hobbies in each table.
>There are three queries I wish to run.
>1) Find all the Hobbie and Name combinations in Table B not in Table A
>only for Names that exist in Table A
>2) Find all Hobbies and Name combinations in Table A not in Table B
>3) Return all data in Table B that contains a Name that exists in Table
>A
>Regards,
>Ciarn

Hi Ciarn,

I guess the real question is why you store the same data in two tables.

Try if the following work. If not, then post table structure and sample
data as described here: www.aspfaq.com/5006.

1)
SELECT b.Hobbie, b.Name
FROM TableB AS b
WHERE EXISTS (SELECT *
FROM TableA AS a
WHERE a.Name = b.Name)

2)
SELECT a.Hobbie, a.Name
FROM TableA AS a
WHERE NOT EXISTS (SELECT *
FROM TableB AS b
WHERE b.Name = a.Name
AND b.Hobbie = a.Hobbie)

3) Same as 1.

Notes:
* 1 and 3 can also be done with an inner join, but you need some way to
prevent duplicates - either DISTINCT or a derived table
* 2 can also be done with an outer join.

Best, Hugo
--

(Remove _NO_ and _SPAM_ to get my e-mail address)|||1 and 2:

SELECT A.name, A.hobby, B.name, B.hobby
FROM TableA AS A
FULL JOIN TableB AS B
ON A.name = B.name
AND A.hobby = B.hobby
WHERE A.name IS NULL
OR B.name IS NULL

3.
SELECT DISTINCT B.name, B.hobby
FROM TableA AS A
JOIN TableB AS B
ON A.name = B.name

Info on the Most Valuable Professional programme:
http://mvp.support.microsoft.com/

--
David Portas
SQL Server MVP
--|||I've tried those queries, but cannot get what I want.
Below are tables like the Tables I am using.

For Query1 I want to return
NameHobby
PhilBoxing
AndyBoxing
CiaranBoxing

For Query2 I want to return
NameHobby
PhilAthletics
AndyRugby
CiaranFootball

For Query3 I want to return
NameHobby
PhilFootball
PhilRugby
PhilAthletics
AndyFootball
AndyRugby
AndyAthletics
CiaranFootball
CiaranRugby
CiaranAthletics

TableA

NameHobby
PhilFootball
PhilRugby
AndyAthletics
AndyFootball
CiaranRugby
CiaranAthletics
PhilBoxing
AndyBoxing
CiaranBoxing

TableB
NameHobby
PhilFootball
PhilRugby
PhilAthletics
AndyFootball
AndyRugby
AndyAthletics
CiaranFootball
CiaranRugby
CiaranAthletics
MarkFootball
MarkRugby
MarkAthletics

Regards,
Ciarn|||I've tried those queries, but cannot get what I want.
Below are tables like the Tables I am using.

For Query1 I want to return
Name Hobby
Phil Boxing
Andy Boxing
Ciaran Boxing

For Query2 I want to return
Name Hobby
Phil Athletics
Andy Rugby
Ciaran Football

For Query3 I want to return
Name Hobby
Phil Football
Phil Rugby
Phil Athletics
Andy Football
Andy Rugby
Andy Athletics
Ciaran Football
Ciaran Rugby
Ciaran Athletics

TableA

Name Hobby
Phil Football
Phil Rugby
Andy Athletics
Andy Football
Ciaran Rugby
Ciaran Athletics
Phil Boxing
Andy Boxing
Ciaran Boxing

TableB
Name Hobby
Phil Football
Phil Rugby
Phil Athletics
Andy Football
Andy Rugby
Andy Athletics
Ciaran Football
Ciaran Rugby
Ciaran Athletics
Mark Football
Mark Rugby
Mark Athletics

Regards,
Ciarn|||I think you swapped the first two around from your original but thanks
for the examples. Try:

SELECT B.name, B.hobby
FROM TableB AS B
LEFT JOIN TableA AS A
ON A.name = B.name
AND A.hobby = B.hobby
WHERE A.hobby IS NULL
AND EXISTS
(SELECT *
FROM TableA
WHERE name = B.name)

SELECT A.name, A.hobby
FROM TableA AS A
LEFT JOIN TableB AS B
ON A.name = B.name
AND A.hobby = B.hobby
WHERE B.hobby IS NULL

My third query seems to produce the result you asked for (in a
different order maybe but you just need to add an ORDER BY clause if a
specific order is important). Try it again and let me know. Here's your
sample in code so that you can easily reproduce it. It's always best to
post your table structures and sample data as code so that others can
understand what your tables look like and test out possible solutions:

CREATE TABLE TableA (name VARCHAR(20) NOT NULL, hobby VARCHAR(20) NOT
NULL, PRIMARY KEY (name,hobby))

CREATE TABLE TableB (name VARCHAR(20) NOT NULL, hobby VARCHAR(20) NOT
NULL, PRIMARY KEY (name,hobby))

INSERT INTO TableA (name, hobby)
SELECT 'Phil', 'Football' UNION ALL
SELECT 'Phil', 'Rugby' UNION ALL
SELECT 'Andy', 'Athletics' UNION ALL
SELECT 'Andy', 'Football' UNION ALL
SELECT 'Ciaran', 'Rugby' UNION ALL
SELECT 'Ciaran', 'Athletics' UNION ALL
SELECT 'Phil', 'Boxing' UNION ALL
SELECT 'Andy', 'Boxing' UNION ALL
SELECT 'Ciaran', 'Boxing'

INSERT INTO TableB (name, hobby)
SELECT 'Phil', 'Football' UNION ALL
SELECT 'Phil', 'Rugby' UNION ALL
SELECT 'Phil', 'Athletics' UNION ALL
SELECT 'Andy', 'Football' UNION ALL
SELECT 'Andy', 'Rugby' UNION ALL
SELECT 'Andy', 'Athletics' UNION ALL
SELECT 'Ciaran', 'Football' UNION ALL
SELECT 'Ciaran', 'Rugby' UNION ALL
SELECT 'Ciaran', 'Athletics' UNION ALL
SELECT 'Mark', 'Football' UNION ALL
SELECT 'Mark', 'Rugby' UNION ALL
SELECT 'Mark', 'Athletics'

Hope this helps.

--
David Portas
SQL Server MVP
--|||Absolutely perfect.
Very much appreciated.
Thanks,
Ciarn

Tuesday, March 6, 2012

2005-Problem restoring database with encrypted columns to diff ser

I need to start encrypting several fields in a database and have been doing
some testing with a test database first. I've run into problems when
attempting to restore the database on either the same server (but different
database) or to a separate server.
First, here's how i created the symmetric key and encrypted data in the
original database:
create master key
encryption by password = 'testAppleA3';
I also saw a posting here where it referenced running the below first:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'testAppleA3';
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;
CLOSE MASTER KEY;
However when i then try to open the key it stil gives an error:
An error occurred during decryption.
create certificate test
with subject = 'test certificate',
EXPIRY_DATE = '1/1/2010';
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
open symmetric key sk_Test decryption by certificate test;
insert into employees values (101,'Jane
Doe',encryptbykey(key_guid('sk_Test'),'$
200000'));
insert into employees values(102,'Bob
Jones',encryptbykey(key_guid('sk_Test'),
'$500000'));
select * from employees
--delete from employees
select id,name,cast(decryptbykey(salary) as varchar(10)) as salary from
employees
close all symmetric keys
Next I backup up this test database and restore it to a new database on a
different server (same issue if restore to different database but on same
server).
Then if i attempt to open the key in the new database and decrypt:
open symmetric key sk_Test decryption by certificate test;
I get the error: An error occurred during decryption.
Ok, well not unexpected, so reading the forums, i try doing the below first
in the new database:
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
Then I try opening the key again and get the error again:
An error occurred during decryption.
So then it occurs to me, maybe i need to drop and recreate it so i do
drop symmetric key sk_test
then
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
and then try to open it.
Same error!
So then i decide, let's drop everything, the master key, the certificate and
then symmetric key:
drop symmetric key sk_test
drop certificate test
drop master key
Then recreate the master key:
create master key
encryption by password = 'testAppleA3';
Restore the certificate from a backup i had made to a file:
CREATE CERTIFICATE test
FROM FILE = 'c:\storedcerts\encryptiontestcert'
Recreate the symmetric key again:
create symmetric key sk_Test
with algorithm = triple_des
encryption by certificate test;
And now open the key only to get the error:
Cannot decrypt or encrypt using the specified certificate, either because it
has no private key or because the password provided for the private key is
incorrect.
So what am I doing wrong here? In this scenario I would appear to have lost
all access to decrypt the data in the database despite restoring from a
backup which restored the symmetric key and certificate and i obviously know
the password for the master key.
I also tried running the command
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
again but this does not resolve the issue.So now I'm confused.
Where this did not work is when i created the intial database using SQL
Server Express edition on a Win XP workstation and then was restoring to a
database located on a SQL Server Standard edition on a Windows 2003 Server.
I just tried creating everything from scratch on one WIndows 2003 Server
with SQL Server Standard edition and then restoring that database to a
different Windows 2003 Server also with SQL Server Standard and this time it
worked as long as i did the below first:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'testAppleA3';
ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;
CLOSE MASTER KEY;
My question (comes to reliabilty concerns) is why did it not work when i did
:
Orig database SQLSvr Express on Win XP
Restore to SQL Svr Std on Win2003Svr
But did work when I did
Orig database SQL Svr Std on Win 2003Svr
Restore to different SQL Svr Std on different Win 2003 Svr
What would the reason for this be?
Thx.
"Robert" wrote:

> I need to start encrypting several fields in a database and have been doin
g
> some testing with a test database first. I've run into problems when
> attempting to restore the database on either the same server (but differen
t
> database) or to a separate server.
> First, here's how i created the symmetric key and encrypted data in the
> original database:
> create master key
> encryption by password = 'testAppleA3';
> I also saw a posting here where it referenced running the below first:
> OPEN MASTER KEY DECRYPTION BY PASSWORD = 'testAppleA3';
> ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;
> CLOSE MASTER KEY;
> However when i then try to open the key it stil gives an error:
> An error occurred during decryption.
>
> create certificate test
> with subject = 'test certificate',
> EXPIRY_DATE = '1/1/2010';
> create symmetric key sk_Test
> with algorithm = triple_des
> encryption by certificate test;
> open symmetric key sk_Test decryption by certificate test;
> insert into employees values (101,'Jane
> Doe',encryptbykey(key_guid('sk_Test'),'$
200000'));
> insert into employees values(102,'Bob
> Jones',encryptbykey(key_guid('sk_Test'),
'$500000'));
> select * from employees
> --delete from employees
> select id,name,cast(decryptbykey(salary) as varchar(10)) as salary from
> employees
> close all symmetric keys
> Next I backup up this test database and restore it to a new database on a
> different server (same issue if restore to different database but on same
> server).
> Then if i attempt to open the key in the new database and decrypt:
> open symmetric key sk_Test decryption by certificate test;
> I get the error: An error occurred during decryption.
> Ok, well not unexpected, so reading the forums, i try doing the below firs
t
> in the new database:
> ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
> Then I try opening the key again and get the error again:
> An error occurred during decryption.
> So then it occurs to me, maybe i need to drop and recreate it so i do
> drop symmetric key sk_test
> then
> create symmetric key sk_Test
> with algorithm = triple_des
> encryption by certificate test;
> and then try to open it.
> Same error!
> So then i decide, let's drop everything, the master key, the certificate a
nd
> then symmetric key:
> drop symmetric key sk_test
> drop certificate test
> drop master key
> Then recreate the master key:
> create master key
> encryption by password = 'testAppleA3';
> Restore the certificate from a backup i had made to a file:
> CREATE CERTIFICATE test
> FROM FILE = 'c:\storedcerts\encryptiontestcert'
> Recreate the symmetric key again:
> create symmetric key sk_Test
> with algorithm = triple_des
> encryption by certificate test;
> And now open the key only to get the error:
> Cannot decrypt or encrypt using the specified certificate, either because
it
> has no private key or because the password provided for the private key is
> incorrect.
> So what am I doing wrong here? In this scenario I would appear to have lo
st
> all access to decrypt the data in the database despite restoring from a
> backup which restored the symmetric key and certificate and i obviously kn
ow
> the password for the master key.
> I also tried running the command
> ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY
> again but this does not resolve the issue.
>|||Hello Robert,
I was not able to reproduce the issue on my side.
On a SQL Express instance, create a database/table, Master key,
Certificate, symmetric key and insert the record with encrypted column.
Backup/restore to the SQL Enterprise edition instance. I could run the
following query successfully on the restored database:
OPEN MASTER KEY DECRYPTION BY PASSWORD = 'testAppleA3';
open symmetric key sk_Test decryption by certificate test;
It seems the issue is related to SQL service account. Is the service
account the same on both SQL Express and SQL Enterprise instances?
Master key protected by old service account credentials may need to be
protected by the current one. You can use the alter service master key
statement to enter the old credentials and have it be protected by the
current account.
Also, you may want to use same Windows account to start SQL instances to
test the situation.
Best Regards,
Peter Yang
MCSE2000/2003, MCSA, MCDBA
Microsoft Online Partner Support
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
========================================
=============
Business-Critical Phone Support (BCPS) provides you with technical phone
support at no charge during critical LAN outages or "business down"
situations. This benefit is available 24 hours a day, 7 days a week to all
Microsoft technology partners in the United States and Canada.
This and other support options are available here:
BCPS:
https://partner.microsoft.com/US/te...erview/40010469
Others: https://partner.microsoft.com/US/te...upportoverview/
If you are outside the United States, please visit our International
Support page:
http://support.microsoft.com/defaul...rnational.aspx.
========================================
=============
This posting is provided "AS IS" with no warranties, and confers no rights.
--
>Thread-Topic: 2005-Problem restoring database with encrypted columns to
diff ser
>thread-index: AcYgUZjYietJilQUTrOvZ82WtyHCWw==
>X-WBNR-Posting-Host: 159.37.7.48
>From: examnotes <robertv@.noemail.nospam>
>References: <4CB8E9AA-99DF-44A1-8375-BDAB8B69A586@.microsoft.com>
>Subject: RE: 2005-Problem restoring database with encrypted columns to
diff ser
>Date: Mon, 23 Jan 2006 11:17:05 -0800
>Lines: 156
>Message-ID: <E0B161AC-6911-45DE-8100-C42FA028BB1F@.microsoft.com>
>MIME-Version: 1.0
>Content-Type: text/plain;
> charset="Utf-8"
>Content-Transfer-Encoding: 7bit
>X-Newsreader: Microsoft CDO for Windows 2000
>Content-Class: urn:content-classes:message
>Importance: normal
>Priority: normal
>X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.0
>Newsgroups: microsoft.public.sqlserver.security
>NNTP-Posting-Host: TK2MSFTNGXA03.phx.gbl 10.40.2.250
>Path: TK2MSFTNGXA02.phx.gbl!TK2MSFTNGP08.phx.gbl!TK2MSFTNGXA03.phx.gbl
>Xref: TK2MSFTNGXA02.phx.gbl microsoft.public.sqlserver.security:26209
>X-Tomcat-NG: microsoft.public.sqlserver.security
>So now I'm confused.
>Where this did not work is when i created the intial database using SQL
>Server Express edition on a Win XP workstation and then was restoring to a
>database located on a SQL Server Standard edition on a Windows 2003 Server.
>I just tried creating everything from scratch on one WIndows 2003 Server
>with SQL Server Standard edition and then restoring that database to a
>different Windows 2003 Server also with SQL Server Standard and this time
it
>worked as long as i did the below first:
>OPEN MASTER KEY DECRYPTION BY PASSWORD = 'testAppleA3';
>ALTER MASTER KEY ADD ENCRYPTION BY SERVICE MASTER KEY;
>CLOSE MASTER KEY;
>My question (comes to reliabilty concerns) is why did it not work when i
did:
>Orig database SQLSvr Express on Win XP
>Restore to SQL Svr Std on Win2003Svr
>But did work when I did
>Orig database SQL Svr Std on Win 2003Svr
>Restore to different SQL Svr Std on different Win 2003 Svr
>What would the reason for this be?
>Thx.
>
>"Robert" wrote:
>
doing[vbcol=seagreen]
different[vbcol=seagreen]
a[vbcol=seagreen]
same[vbcol=seagreen]
first[vbcol=seagreen]
and[vbcol=seagreen]
because it[vbcol=seagreen]
is[vbcol=seagreen]
lost[vbcol=seagreen]
know[vbcol=seagreen]
>

Saturday, February 25, 2012

2005 too fast "problem"!

I have a weird situation I had not expected....

I insert a record to a table and "later" I update it.
I have two fields to capture time information: Created and LastModified.
My update is very simple: update ... set ..,[LastModifiedDate] = GetDate() where id = @.pId.

Now my problem is that I am seeing the created and lastmodified times as the same (in format 2007-09-05 12:38:42.383) !!??!

The record has definitely been updated (other fields are populated).

Can anybody enlighten me?You'll need to post your code and tell us how long the duration is between inserts & updates.|||Well...

Maybe, upon the selection of the row, you should also grab the modifiedDate as well as the ID, and then check upon update if the dates are the same, otherwise it means someone has cheanged on you since you last grabbed the row...meaning it would appear a miracle occured.

Now this may not be your problem, but it is definetly a doubt, becausde you can't tell, and I'll always try to code toward absolut...I mean absolutes

Here's a sample

UPDATE [MEP]
SET
MEPType = @.MEPType
, MEPName = @.MEPName
, MEPStatus = @.MEPStatus
, MEPStatusDate = @.MEPStatusDate
, ProjectNum = @.ProjectNum
, PropCompDate = @.PropCompDate
, Comments = @.Comments
, ModifiedDate = GetDate()
, ModifiedBy = @.APPUSER

WHERE MEPRecID = @.MEPRecID
AND ModifiedDate = @.ModifiedDate

SELECT @.error = @.@.ERROR, @.rowcount = @.@.ROWCOUNT

IF @.error <> 0
BEGIN
SELECT @.Error_Loc = 1, @.Error_Type = 50001, @.rc = -1
GOTO usp_UPD_MEP_Err
END

IF @.rowcount = 0
BEGIN
IF NOT EXISTS(SELECT * FROM [MEP]
WHERE [MEPRecID]=@.MEPRecID)
BEGIN
SELECT @.Error_Loc = 2, @.Error_Type = 50002, @.rc = 1
, @.Error_Message = 'No Rows Updated. Data has been deleted or ID passed is incorrect'
GOTO usp_UPD_MEP_Err
END
ELSE
BEGIN
SELECT @.Error_Loc = 3, @.Error_Type = 50002, @.rc = 2
, @.Error_Message = 'Rows Have Been Modified since last SELECT'
GOTO usp_UPD_MEP_Err
END
END|||This is a problem? Man, I need a new job.|||This is part of a bigger system so posting code would be confusing but I guess you are suggesting I try to isolate a simple instance of the issue. The operation definitely occurs in 2 distinct steps :

1. insert record and raise an event to tell a listener that the record is available (C# stuff)
2. listener does its thing and updates the record.

I suppose my question really was whether you folks are surprised by this - or whether you believe it you be possible at all? I certainly have put a lot of effort into the performance of this piece of code (real-time system processing 1000s messages from queues, applying them to business data, raising distributed events to remote desktops, etc.) but sure don't believe that it can do even the simplest insert followed by update within the same millisecond!|||Ummmmmm...are you using stored procedures?

Raising an error so you can know, for every row that's
inserted...and you think that's fast|||sure don't believe that it can do even the simplest insert followed by update within the same millisecond!Easily. Try writing a loop to insert GETDATE into a table. You will get a fair few identical records. BTW - the smallest unit of a SQL Server datetime data type is 3.3 milliseconds - maybe you can believe it can do this in 3 times the time you thought it was ;)

The key point is the amount of time between issuing the two instructions. If this is > 3.3 milliseconds then your code is relevent because it must be wrong.|||Are you just trying to get the actual millisecons the row was inserted?|||I fully believe SQL 2005 is this fast. My time trials back in 2004 showed close to 100% improvement over SQL 2000, but it was not a very scientific test. If you need the extra precision, I am afraid you will have to wait until next summer when SQL 2008 comes out, and you can use the new date and time types.|||I trialled a datetime column as a clustered PK inserting getdate in a loop in 2K. Pk violations so it was this fast before 2005.|||Thanks for those responses.

Obviously I'm not a regular user of this forum so I'm not sure of the how you folks communicate/interact so I'll try to clarify.

Brett:

Maybe, upon the selection of the row, you should also grab the modifiedDate as well as the ID,...

..thing is that I don't have any concurrency issue with this. Just an unexpected situation since upgrading to 2005 (i.e. I have never seen this happen in 2000).

MCrowley:

This is a problem? Man, I need a new job.

..:) only problem is that I didn't ever expect such a situation. I do, however, use the Create = LastModified in some logic for lack of another field that would tell me if any attempt at all has been made to process this record. The only field/flag I have is a 'Processed' one but it means whether the data was successfully applied to the business data or not.

Probably not explaining that so well - basically the records I'm talking about are xml messages coming off queues. There are three very distinct steps once a message arrives - 1. persist it locally (withing a queue transaction) and only then 2. process it - meaning use its data to update various other data in the database and 3. broadcast the new 'business' data (business objects really) to all desktops. Important point is the taking form the queue and putting in the data is one distinct transactioned step. Processing it an updating the LastModified flag is another so I would have expected at least a millisecond to have passed.

Brett:

Ummmmmm...are you using stored procedures? Raising an error so you can know, for every row that's inserted...and you think that's fast

..not understanind this post at all - sorry.

Pootle:

Easily. Try writing a loop to insert GETDATE into ...

..yep, fair enough I would expect that. But this is two distinct (almost unrelated) stored procedure calls from two distinct processed!?

The key point is the amount of time between issuing the two instructions

..the only measure I have of this is those timestamps i'm afraid.

Brett:

Are you just trying to get the actual millisecons the row was inserted?

..No, see above response to MCrowley - I have used the "fact" that they could never be the same to provide some other logic. Bad design and all but I was limited to the existing table structure.

Just to reiterate: This is NOT any kind of major problem. Just a curiousity. It happens maybe a 5 times a day and usually when processing the contents of the message is expected to be very fast anyway - for example, when I immediated find that some content is bad.|||I'm not sure what the two distinct processes are. If you mean these are 3GL apps calling SQL Server and therer is communication in between the two statements then that sound odd. If it is all within SQL server then, irrespective of how many sproc calls there are, you would expect it to be lightening fast. Again - it all depends what goes on in between. I'm afraid I don't understand your business processes enough to get a handle on your description above.

You could add the below between the two statements to be absolutely satisfied what is going on.

WAITFOR DELAY '00:00:00:002'|||Pootle's suggestion was what I was thinking of, if this turned out to be a problem. 5 - 10 rows out of 1,000's per day is not a big problem...except for the guy who has to fix 5 - 10 rows per day. i would only add to Pootle's suggestion by adding a check to the update procedure something like

if CreatedDate = getdate()
begin
WAITFOR DELAY '00:00:00:002'
end
update stuff goes here

This might save you even more devious problems later on.|||5 - 10 rows out of 1,000's per day is not a big problem...except for the guy who has to fix 5 - 10 rows per day. Lol. That is so true. Even after years of working with sets I still sometimes forget that it is the number of distinct problems rather than the sum number of rows affected that determine how much effort I need to put into debugging\ cleansing etc..|||Are you just trying to get the actual millisecons the row was inserted?

..No, see above response to MCrowley - I have used the this "fact" that they could never be the same to provide some other logic.

Holy Cow...what would that "Logic" (Logic should be in quotes) be?

Bad design and all but I was limited to the existing table structure.

Good you think that's not a good idea, but what does it mean limites to the table structure?

And if you are limited, I would suggest triggers to fix it|||Again thanks for taking the time to reply.

Brett:
Holy Cow...what would that "Logic" (Logic should be in quotes) be?

.."fact" (in quotes) - I was being ironic(?)

Good you think that's not a good idea, but what does it mean limites to the table structure?

..should have be written limiteD meaning i couldn't change the structure to add a new field.

Generally speaking I get the vibe that you folks haven't really had any issue with this kind of stuff. I don't have either - if the update time is the same as the insert time and i can clearly see from the data that they did not/could not occur in a single transaction/statement/whatever then so be it.

Just fyi in case it all seems crazy to you...

..messages sent from other systems to mine are read from various incoming protocols - queues, ftp, etc. using a message manager process (threading/processors/etc.)

..ideally the data they contain are applied directly to the appropriate database tables, etc. BUT for safety sake (and believe me with good reason) I persist every piece of incoming data UNTIL it is processed successfully.

..if persisting a message to the database is successful that particular transaction is committed (in the case of a queue the message is truly removed). Otherwise everything is rolled back.

..that essentially is 1 process inserting some 'raw' data.

..then the component that has read the message and persisted it to the database raises an event to let any interested parties that a message has come in and is available in the message repository using Id xxx.

..any listeners then do whatever they need to do with the message to make it into something meaningful to that particular application.

..in my case here there is only one listener. It gets the event and the Id and calls a stored procedure which takes the Id, reads the message, processes its contents and finally (finally!!) updates the message with results from the processing - example: Processed = false, Reason = 'cannot find this account/etc.'

....now obviously there is a significant overhead in persisting the raw messages but the nature (trustworthyness - i.e. "we never sent that") of the other systems and the delivery protocols means that it must be done.

.. so you can see that the two processes/actions are very distinct. That is why I am amazed that the times are the same. But the proof is there in the database - the insert uses getdate() and the update uses getdate() and the update has clearly occurred (Failure Reason is populated).

I'm damn happy that the performance is so good to be perfectly honest. I just have difficulty believing it is all.

And, again, to reiterate there is no problem - i.e. the times matching is NOT a problem. Its just a curiousity. The waitfor idea is a great suggestion should I decide that I don't want these times to ever be the same. But, then, for performance stats it really helps out that they sometimes are :)

Again, thanks all for taking an interest.

-- also sorry for my bad spelling/clumsy fingers

Thursday, February 16, 2012

2005 Inconsistent Results with VarBinary fields

Hello,
I seem to be encountering some inconsistent results with full text searching
against a combination of VarChar(max) and VarBinary(max) fields.
I have a table which has both a VarChar(max) field and a VarBinary(max)
field. The table is meant to either have text or an attachement (or both).
I have indexed both fields. When I do a particular search against just the
Varbinary field I get 16 results. When I do the same query, but with both
columns specified, I only get 8 results. Maybe there is a good reason for
this, but it seems like adding more fields to the search should not result
in fewer hits. And I have verified that the records it is missing truly
contain the word I am looking for. Below are the relevant queries.
Any help would be appreciated. Thanks!
Randall
-- ***** Table ****
create table dbo.z_FullTextChild (
FullTextChildID uniqueidentifier ROWGUIDCOL NOT NULL default newid(),
FullTextMainID uniqueidentifier not null,
LongText VarChar(max) null,
AttachmentFileName VarChar(256) null,
AttachmentFileType VarChar(15) null,
Attachment VarBinary(max),
constraint PK_FullTextChild primary key (FullTextChildID),
constraint PK_FullTextChild_FullTextMain foreign key (FullTextMainID)
references dbo.z_FullTextMain(FullTextMainID) on update cascade on delete
cascade
)
go
-- ******** Full Text Index *********
create fulltext index on dbo.z_FullTextChild
(LongText, Attachment TYPE COLUMN AttachmentFileType )
key index PK_FullTextChild
on ftTracker
with CHANGE_TRACKING AUTO
go
-- ******* Query that yields 16 results ***************
select ftc.LongText, ftc.AttachmentFileName, ft.Rank
from z_FullTextChild ftc
join freetexttable (z_FullTextChild, (Attachment), 'INFORMATION', 100) ft
on ft.[KEY]=ftc.FullTextChildID
where
ftc.AttachmentFileName is not null
order by ft.Rank desc
-- ******** Query that yields 8 results (only difference is in selected
columns) ******************
select ftc.LongText, ftc.AttachmentFileName, ft.Rank
from z_FullTextChild ftc
join freetexttable (z_FullTextChild, (Attachment, LongText), 'INFORMATION',
100) ft on ft.[KEY]=ftc.FullTextChildID
where
ftc.AttachmentFileName is not null
order by ft.Rank desc
-- ******* Select @.@.Version ***********
Microsoft SQL Server 2005 - 9.00.1187.07 (Intel X86) May 24 2005 18:22:46
Copyright (c) 1988-2005 Microsoft Corporation Beta Edition on Windows NT
5.1 (Build 2600: Service Pack 2)
I have tried to repro this problem and am unable to do so. Can you perhaps
do this query?
select ftc.LongText, ftc.AttachmentFileName, ft.Rank
from z_FullTextChild ftc
join freetexttable (z_FullTextChild, (LongText), 'INFORMATION',
100) ft on ft.[KEY]=ftc.FullTextChildID
where
ftc.AttachmentFileName is not null
order by ft.Rank desc
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Randall" <randall@.randall.com> wrote in message
news:uYYBzNuhFHA.2472@.TK2MSFTNGP15.phx.gbl...
> Hello,
> I seem to be encountering some inconsistent results with full text
searching
> against a combination of VarChar(max) and VarBinary(max) fields.
> I have a table which has both a VarChar(max) field and a VarBinary(max)
> field. The table is meant to either have text or an attachement (or
both).
> I have indexed both fields. When I do a particular search against just
the
> Varbinary field I get 16 results. When I do the same query, but with both
> columns specified, I only get 8 results. Maybe there is a good reason for
> this, but it seems like adding more fields to the search should not result
> in fewer hits. And I have verified that the records it is missing truly
> contain the word I am looking for. Below are the relevant queries.
> Any help would be appreciated. Thanks!
> Randall
> -- ***** Table ****
> create table dbo.z_FullTextChild (
> FullTextChildID uniqueidentifier ROWGUIDCOL NOT NULL default newid(),
> FullTextMainID uniqueidentifier not null,
> LongText VarChar(max) null,
> AttachmentFileName VarChar(256) null,
> AttachmentFileType VarChar(15) null,
> Attachment VarBinary(max),
> constraint PK_FullTextChild primary key (FullTextChildID),
> constraint PK_FullTextChild_FullTextMain foreign key (FullTextMainID)
> references dbo.z_FullTextMain(FullTextMainID) on update cascade on delete
> cascade
> )
> go
> -- ******** Full Text Index *********
> create fulltext index on dbo.z_FullTextChild
> (LongText, Attachment TYPE COLUMN AttachmentFileType )
> key index PK_FullTextChild
> on ftTracker
> with CHANGE_TRACKING AUTO
> go
> -- ******* Query that yields 16 results ***************
> select ftc.LongText, ftc.AttachmentFileName, ft.Rank
> from z_FullTextChild ftc
> join freetexttable (z_FullTextChild, (Attachment), 'INFORMATION', 100) ft
> on ft.[KEY]=ftc.FullTextChildID
> where
> ftc.AttachmentFileName is not null
> order by ft.Rank desc
> -- ******** Query that yields 8 results (only difference is in selected
> columns) ******************
> select ftc.LongText, ftc.AttachmentFileName, ft.Rank
> from z_FullTextChild ftc
> join freetexttable (z_FullTextChild, (Attachment, LongText),
'INFORMATION',
> 100) ft on ft.[KEY]=ftc.FullTextChildID
> where
> ftc.AttachmentFileName is not null
> order by ft.Rank desc
> -- ******* Select @.@.Version ***********
> Microsoft SQL Server 2005 - 9.00.1187.07 (Intel X86) May 24 2005
18:22:46
> Copyright (c) 1988-2005 Microsoft Corporation Beta Edition on Windows NT
> 5.1 (Build 2600: Service Pack 2)
>
>
|||Thanks for the reply.
That particular query yields no hits because currently none of the records
that have attachments have any LongText associated with them (although that
could change in the future).
Randall
"Hilary Cotter" <hilary.cotter@.gmail.com> wrote in message
news:O4eEKfuhFHA.1048@.tk2msftngp13.phx.gbl...
>I have tried to repro this problem and am unable to do so. Can you perhaps
> do this query?
> select ftc.LongText, ftc.AttachmentFileName, ft.Rank
> from z_FullTextChild ftc
> join freetexttable (z_FullTextChild, (LongText), 'INFORMATION',
> 100) ft on ft.[KEY]=ftc.FullTextChildID
> where
> ftc.AttachmentFileName is not null
> order by ft.Rank desc
>
> --
> Hilary Cotter
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Randall" <randall@.randall.com> wrote in message
> news:uYYBzNuhFHA.2472@.TK2MSFTNGP15.phx.gbl...
> searching
> both).
> the
> 'INFORMATION',
> 18:22:46
>