Showing posts with label delivery. Show all posts
Showing posts with label delivery. Show all posts

Saturday, February 25, 2012

2005 transactional replication statement delivery

Hi There

WHen creating publciations under 2005 i saw a very interesting option under stament delivery, for inserts , updates , deletes there is an option that simply says insert/update/delete statement.

I could find very little in BOL about this under "Article Properties", is this what it soudns like ? FOr example if you say :

update Table set Cloumn = 'whatever', this will not trigger the update sp for each row at the subscriber , will it actually deliver the update statements and literally do the update/insert/delete statement at the subscriber?

Thanx

Yes. It is equivalent to supply value 'SQL' for parameter @.ins_cmd/@.upd_cmd/@.del_cmd in sp_addarticle. For more information, please take a look at the document for sp_addarticle http://msdn2.microsoft.com/en-us/library/ms173857.aspx.

Peng

|||just an FYI, using SQL instead of the custom stored procedures has a negative perf impact, it's primarily used for non-SQL server subscribers. Stick with the custom stored procs if you can.|||

HI Greg

That is a very interresting point you bring up.

I was thinking of it exactly for performance, often in replication we perfom an update statement on the publisher that may take 10 minutes to execute and update say 2 millions rows. However this causes replication latency of up to half an hour at the subscriber at the execution of the MSupd_repl proc 2 million times is slow sometimes each execution of the proc can take a tenth of a second which is alot when you have millions of rows to update, performing the same update at the subscriber would take 10 minutes.

So i actually thought this option could help but apparantly not ?

|||

Hi Dietz,

Change delivery method to "SQL" won't help the performance as replication will still deliver the changes row by row. It means that, in your example, we still call update 2 million times. To solve this problem, you can utilize the SQL 2005 functionality "Publishing Stored procedure Execution in Transactional Replication". More info can be found here (http://msdn2.microsoft.com/en-us/library/ms152754.aspx). It will require you to use stored procedures for your update though.

Peng

|||

The performance you're seeing has nothing to do with SQL vs custom stored procs. In fact, if you used SQL, I bet it owuld be much slower since with SQL, multiple commands are batched then sent via one parameterized statement. With custom stored procs, the proc is compiled once, with SQL, you're pretty much guaranteed that no two batches will be the same, thus you'll have a ton of compilations.

What you're experiencing can be worked around by replicating the execuction of a stored procedure. http://msdn2.microsoft.com/en-us/library/ms152754.aspx.

|||

Thanx Pend and Greg for the feedback, but how is this different from the usual method if it will still call update 2 millios times. DO you mean that instead of calling the MSupd sp on the sunscriber it delivers and update statement for each row?

I am aware of the sp publication method which was available in 2000 i thought this was somehow a similar concept. Thanx

|||If you issue a single update statement at the publisher that affects 2 million rows, you will get 2 million update statements, one for each row, replicated to the subscriber regardless whether you use SQL or custom stored procedures. If you do these batches often, not only can you experience long latency but your distribution database will fill up quickly. In these cases, it's recommended to replicate the execution of a stored procedure if latency and disk space is an issue. By putting your update statement in a proc, and replicating proc execution, rather than send 2 million deletes, it sends the proc call to the subscriber. Now you only have one command instead of 2 million.|||

Hi Greg

Yes that is wagt i thought, however we are getting rid of replication, but good to know.

Thanx

Sunday, February 19, 2012

2005 Merge Agent Degradation

I have a merge agent running extremely slow - schedule fires every 1 minute,
however it takes 1 to 3 minutes to execute. Delivery Rate 5 rows/sec or less,
all 5 other merge agents execute in less than 20 seconds and Delivery Rate 80
rows/sec or greater.
Each time merge agent fires, subscr cpu spikes to 85% - normally 15 to 20%.
Ran profiler and discovered this stamement below hogging the cpu and having
the longest duration.
I am rebuilding the MSmerge_genhistory table nightly w/ the following: DBCC
DBREINDEX (MSmerge_genhistory, '', 80)
Any ideas? A re-init will resolve problem , have had to do this many times
in past - at least every 30 days... however, want to get to root cause. tia
Chris
select top (@.numgens) *
from
(
select generation, guidsrc, art_nick,
case when genstatus = 4 then 0 else genstatus end as
genstatus,
pubid, nicknames,
okaytoskip = case when
art_nick is not null and art_nick <> 0
and genstatus in (0,4)
-- Skip all rows that are for incomplete
generations for articles that have no joins.
and not exists (select 1 from
dbo.sysmergesubsetfilters where (join_nickname = art_nick or art_nickname =
art_nick) and (filter_type & 1) = 1)
then 1 else 0 end
, changecount
from
(select generation, guidsrc, art_nick, genstatus, pubid,
nicknames, changecount
from dbo.MSmerge_genhistory with (rowlock, repeatableread)
where generation >= @.genstart
and generation <= @.maxgen_to_enumerate
and generation > @.mingen_to_enumerate
and (art_nick = 0 or art_nick is NULL or
art_nick in (select nickname from dbo.sysmergearticles
where pubid = @.pubid))
) as generation_range
UNION ALL -- use UNION ALL instead of UNION for perf
reasons. Merge agent code will skip dupes. Will only have max 2 dupes.
select generation = @.next_possible_watermark, guidsrc =
@.next_possible_watermark_guidsrc,
art_nick = @.next_possible_watermark_art_nick, genstatus =
@.next_possible_watermark_genstatus,
pubid = @.next_possible_watermark_pubid, nicknames =
@.next_possible_watermark_nicknames,
okaytoskip = 0, changecount =
@.next_possible_watermark_changecount
where @.next_possible_watermark is not null
union all
select generation = @.min_open_gen, guidsrc= @.min_open_gen_guid,
art_nick=@.min_open_gen_art_nick, genstatus=0,
pubid=@.pubid, nicknames=NULL, okaytoskip = 0,
changecount=0
where @.min_open_gen is not null
) as genertions
order by generation ASC
Two things I can suggest here is:
1. Avoid metadata contention. Make sure merge agents do not run at the
same time so stagger their schedules.
2. Retention period can cause lots of data to be stored in the contets
and tombstone tables. Default i think is 14 days. So 14 days worth of
changes are stored. Consider reducing this if subscribers are not
offline for a long time.
Regards Jim
http://jims-spanakopita.blogspot.com/
|||What do your join filters look like? By placing indexes on all the filters
you will get better performance. Another option to try is to modify your
filters to make them as shallow as possible.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Chris" <Chris@.discussions.microsoft.com> wrote in message
news:4ED9E8B8-9CA1-4B10-A609-E67921202C68@.microsoft.com...
> I have a merge agent running extremely slow - schedule fires every 1
> minute,
> however it takes 1 to 3 minutes to execute. Delivery Rate 5 rows/sec or
> less,
> all 5 other merge agents execute in less than 20 seconds and Delivery Rate
> 80
> rows/sec or greater.
> Each time merge agent fires, subscr cpu spikes to 85% - normally 15 to
> 20%.
> Ran profiler and discovered this stamement below hogging the cpu and
> having
> the longest duration.
> I am rebuilding the MSmerge_genhistory table nightly w/ the following:
> DBCC
> DBREINDEX (MSmerge_genhistory, '', 80)
>
> Any ideas? A re-init will resolve problem , have had to do this many times
> in past - at least every 30 days... however, want to get to root cause.
> tia
> Chris
>
> select top (@.numgens) *
> from
> (
> select generation, guidsrc, art_nick,
> case when genstatus = 4 then 0 else genstatus end as
> genstatus,
> pubid, nicknames,
> okaytoskip = case when
> art_nick is not null and art_nick <> 0
> and genstatus in (0,4)
> -- Skip all rows that are for incomplete
> generations for articles that have no joins.
> and not exists (select 1 from
> dbo.sysmergesubsetfilters where (join_nickname = art_nick or art_nickname
> =
> art_nick) and (filter_type & 1) = 1)
> then 1 else 0 end
> , changecount
> from
> (select generation, guidsrc, art_nick, genstatus, pubid,
> nicknames, changecount
> from dbo.MSmerge_genhistory with (rowlock, repeatableread)
> where generation >= @.genstart
> and generation <= @.maxgen_to_enumerate
> and generation > @.mingen_to_enumerate
> and (art_nick = 0 or art_nick is NULL or
> art_nick in (select nickname from dbo.sysmergearticles
> where pubid = @.pubid))
> ) as generation_range
> UNION ALL -- use UNION ALL instead of UNION for perf
> reasons. Merge agent code will skip dupes. Will only have max 2 dupes.
> select generation = @.next_possible_watermark, guidsrc =
> @.next_possible_watermark_guidsrc,
> art_nick = @.next_possible_watermark_art_nick, genstatus =
> @.next_possible_watermark_genstatus,
> pubid = @.next_possible_watermark_pubid, nicknames =
> @.next_possible_watermark_nicknames,
> okaytoskip = 0, changecount =
> @.next_possible_watermark_changecount
> where @.next_possible_watermark is not null
> union all
> select generation = @.min_open_gen, guidsrc= @.min_open_gen_guid,
> art_nick=@.min_open_gen_art_nick, genstatus=0,
> pubid=@.pubid, nicknames=NULL, okaytoskip = 0,
> changecount=0
> where @.min_open_gen is not null
> ) as genertions
> order by generation ASC
>