[Agilo for Trac] Very slow with many sprints and team members

Discussion:

Torq

2013-09-03 09:37:03 UTC

Viewing sprint backlogs, sprints and editing sprints is very slow.
Viewing a sprint from backlog can take 15 seconds.

We have over 100 sprints in agilo_sprint (past and future)

Single click on ticket from Sprint Backlog generates about 20 000 loglines
like those below.
Essentially every team member's name and email is checked for every sprint.

Removing team from past sprints speeds things up.
This seems like a hack, so I'm interested to know what we are doing wrong
here.

Missconfiguration or bad data in db?

We've used Trac&Agilo for over 4 years and the slowness has crept on us
slowly.
Just to debug this slowness I've installed a new instance on Ubuntu v12
64bit server on Hyper-v with
binary_agilo-1.3.10_PRO-py2.7.egg
postgressql-9.1-main.log
and Radius for authentication.

DB was imported from the old trac&Agilo installation.

Thank you,
Torq

trac.log
2013-09-02 14:51:15,221 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,221 Trac[util] DEBUG: args: [u'user1', 'name']
2013-09-02 14:51:15,222 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,222 Trac[util] DEBUG: args: [u'user1', 'email']
2013-09-02 14:51:15,224 Trac[log] DEBUG: [Sprint]: Returning duration 11
start: 2013-12-02 13:00:00+00:00, end: 2013-12-16 13:00:00+00:00
2013-09-02 14:51:15,224 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,225 Trac[util] DEBUG: args: [u'user2', 'name']
2013-09-02 14:51:15,226 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,226 Trac[util] DEBUG: args: [u'user2', 'email']
2013-09-02 14:51:15,227 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,227 Trac[util] DEBUG: args: [u'user3', 'name']
2013-09-02 14:51:15,229 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,229 Trac[util] DEBUG: args: [u'user3', 'email']
2013-09-02 14:51:15,230 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,230 Trac[util] DEBUG: args: [u'user4', 'name']
2013-09-02 14:51:15,231 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,231 Trac[util] DEBUG: args: [u'user4', 'email']
2013-09-02 14:51:15,232 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,233 Trac[util] DEBUG: args: [u'somebody', 'name']
2013-09-02 14:51:15,234 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,234 Trac[util] DEBUG: args: [u'somebody', 'email']
2013-09-02 14:51:15,235 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,235 Trac[util] DEBUG: args: [u'user5', 'name']
2013-09-02 14:51:15,236 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,236 Trac[util] DEBUG: args: [u'user5', 'email']
2013-09-02 14:51:15,237 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,238 Trac[util] DEBUG: args: [u'user6', 'name']
2013-09-02 14:51:15,239 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,239 Trac[util] DEBUG: args: [u'user6', 'email']
2013-09-02 14:51:15,240 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,240 Trac[util] DEBUG: args: [u'user7', 'name']
2013-09-02 14:51:15,241 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,241 Trac[util] DEBUG: args: [u'user7', 'email']
2013-09-02 14:51:15,242 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,243 Trac[util] DEBUG: args: [u'user8', 'name']
2013-09-02 14:51:15,244 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,244 Trac[util] DEBUG: args: [u'user8', 'email']
2013-09-02 14:51:15,245 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,245 Trac[util] DEBUG: args: [u'user9', 'name']
2013-09-02 14:51:15,246 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,246 Trac[util] DEBUG: args: [u'user9', 'email']
2013-09-02 14:51:15,247 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,248 Trac[util] DEBUG: args: [u'user10', 'name']
2013-09-02 14:51:15,249 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,249 Trac[util] DEBUG: args: [u'user10', 'email']
2013-09-02 14:51:15,250 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,250 Trac[util] DEBUG: args: [u'user11', 'name']
2013-09-02 14:51:15,251 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,251 Trac[util] DEBUG: args: [u'user11', 'email']
2013-09-02 14:51:15,252 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,253 Trac[util] DEBUG: args: [u'user12', 'name']
2013-09-02 14:51:15,254 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,254 Trac[util] DEBUG: args: [u'user12', 'email']
2013-09-02 14:51:15,255 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,255 Trac[util] DEBUG: args: [u'user13', 'name']
2013-09-02 14:51:15,257 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,257 Trac[util] DEBUG: args: [u'user13', 'email']
2013-09-02 14:51:15,260 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,260 Trac[util] DEBUG: args: [u'user14', 'name']
2013-09-02 14:51:15,261 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,261 Trac[util] DEBUG: args: [u'user14', 'email']
2013-09-02 14:51:15,262 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,263 Trac[util] DEBUG: args: [u'user1', 'name']
2013-09-02 14:51:15,264 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,264 Trac[util] DEBUG: args: [u'user1', 'email']
2013-09-02 14:51:15,265 Trac[log] DEBUG: [Sprint]: Returning duration 11
start: 2013-12-16 13:00:00+00:00, end: 2013-12-30 13:00:00+00:00
2013-09-02 14:51:15,266 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,266 Trac[util] DEBUG: args: [u'user2', 'name']
2013-09-02 14:51:15,267 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,268 Trac[util] DEBUG: args: [u'user2', 'email']
2013-09-02 14:51:15,269 Trac[util] DEBUG: SQL: SELECT value FROM
session_attribute WHERE sid=%s AND name=%s
2013-09-02 14:51:15,269 Trac[util] DEBUG: args: [u'user3', 'name']

postgressql-9.1-main.log
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.180 ms statement: SELECT value
FROM session_attribute WHERE sid='user1' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.180 ms statement: SELECT value
FROM session_attribute WHERE sid='user1' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.183 ms statement: SELECT value
FROM session_attribute WHERE sid='user2' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.182 ms statement: SELECT value
FROM session_attribute WHERE sid='user2' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.200 ms statement: SELECT value
FROM session_attribute WHERE sid='user3' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.183 ms statement: SELECT value
FROM session_attribute WHERE sid='user3' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.182 ms statement: SELECT value
FROM session_attribute WHERE sid='user4' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.181 ms statement: SELECT value
FROM session_attribute WHERE sid='user4' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.195 ms statement: SELECT value
FROM session_attribute WHERE sid='somebody' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.177 ms statement: SELECT value
FROM session_attribute WHERE sid='somebody' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.183 ms statement: SELECT value
FROM session_attribute WHERE sid='user5' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.182 ms statement: SELECT value
FROM session_attribute WHERE sid='user5' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.201 ms statement: SELECT value
FROM session_attribute WHERE sid='user6' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.183 ms statement: SELECT value
FROM session_attribute WHERE sid='user6' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.182 ms statement: SELECT value
FROM session_attribute WHERE sid='user7' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.182 ms statement: SELECT value
FROM session_attribute WHERE sid='user7' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.201 ms statement: SELECT value
FROM session_attribute WHERE sid='user8' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.327 ms statement: SELECT value
FROM session_attribute WHERE sid='user8' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.183 ms statement: SELECT value
FROM session_attribute WHERE sid='user9' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.022 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.207 ms statement: SELECT value
FROM session_attribute WHERE sid='user9' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.034 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.185 ms statement: SELECT value
FROM session_attribute WHERE sid='user10' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.034 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.024 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.242 ms statement: SELECT value
FROM session_attribute WHERE sid='user10' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.032 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.269 ms statement: SELECT value
FROM session_attribute WHERE sid='user11' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.030 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.187 ms statement: SELECT value
FROM session_attribute WHERE sid='user11' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.184 ms statement: SELECT value
FROM session_attribute WHERE sid='user12' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.184 ms statement: SELECT value
FROM session_attribute WHERE sid='user12' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.184 ms statement: SELECT value
FROM session_attribute WHERE sid='user13' AND name='name'
2013-09-02 14:51:15 EEST LOG: duration: 0.026 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
2013-09-02 14:51:15 EEST LOG: duration: 0.184 ms statement: SELECT value
FROM session_attribute WHERE sid='user13' AND name='email'
2013-09-02 14:51:15 EEST LOG: duration: 0.025 ms statement: ROLLBACK
2013-09-02 14:51:15 EEST LOG: duration: 0.023 ms statement: BEGIN
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Stefano Rago

2013-09-10 13:38:27 UTC

Permalink

Hi Torq,

Post by Torq
Viewing sprint backlogs, sprints and editing sprints is very slow.
Viewing a sprint from backlog can take 15 seconds.
We have over 100 sprints in agilo_sprint (past and future)
Single click on ticket from Sprint Backlog generates about 20 000 loglines like those below.
Essentially every team member's name and email is checked for every sprint.

Did you already compare these performances with logging turned off?

Post by Torq
Removing team from past sprints speeds things up.
This seems like a hack, so I'm interested to know what we are doing wrong here.
Missconfiguration or bad data in db?

What is the approximate size of the "ticket_custom" table? How many tuples are there?
In general, could you please provide the tuple count for all of the tables in the environment's database?

Post by Torq
We've used Trac&Agilo for over 4 years and the slowness has crept on us slowly.
Just to debug this slowness I've installed a new instance on Ubuntu v12 64bit server on Hyper-v with
binary_agilo-1.3.10_PRO-py2.7.egg
postgressql-9.1-main.log
and Radius for authentication.
DB was imported from the old trac&Agilo installation.

Would it be possible to compare performances in a condition where Radius is disabled / bypassed?

Stefano Rago
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Torq

2013-09-11 07:40:49 UTC

Permalink

Hi Stefano

Thank you for your reply.

Post by Torq
Single click on ticket from Sprint Backlog generates about 20 000 loglines
like those below.
Essentially every team member's name and email is checked for every sprint.
Did you already compare these performances with logging turned off?
Original production Trac was very slow without logging.

I added the logging for the new debugging instance.
Comparing performance between these two systems is not reliable since
underlying hardware differs.

Post by Torq
Removing team from past sprints speeds things up.
This seems like a hack, so I'm interested to know what we are doing wrong here.
Missconfiguration or bad data in db?
What is the approximate size of the "ticket_custom" table? How many tuples are there?
In general, could you please provide the tuple count for all of the tables
in the environment's database?
Ticket_custom has 13 308 rows

List of all:

nspname | relname | reltuples

I'll have to see about that.
I might be able to access the data via Trac-admin user without the Radius
in between.

Torq
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Stefano Rago

2013-09-16 07:37:55 UTC

Permalink

Hi Torq,

would you mind providing us with the content of your trac.ini file and installed plugins list?

Make sure you remove any sensitive data from the trac.ini file first.

Also, would you be favourable to send us a dump of your database for us to profile the system with this specific workload?
In case you are, you can simply email it to customer.support-CKHYNj9aYvRx67MzidHQgQC/***@public.gmane.org

Cheers

Stefano Rago

Post by Torq
Hi Stefano
Thank you for your reply.

Post by Torq
Single click on ticket from Sprint Backlog generates about 20 000 loglines like those below.
Essentially every team member's name and email is checked for every sprint.

Did you already compare these performances with logging turned off?
Original production Trac was very slow without logging.
I added the logging for the new debugging instance.
Comparing performance between these two systems is not reliable since underlying hardware differs.

Post by Torq
Removing team from past sprints speeds things up.
This seems like a hack, so I'm interested to know what we are doing wrong here.
Missconfiguration or bad data in db?

Would it be possible to compare performances in a condition where Radius is disabled / bypassed?
I'll have to see about that.
I might be able to access the data via Trac-admin user without the Radius in between.
Torq
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum
You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.
To unsubscribe from this group, send an email to
For more options, visit this group at
http://groups.google.com/group/agilo
---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
For more options, visit https://groups.google.com/groups/opt_out.

--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

t***@public.gmane.org

2013-09-17 05:48:47 UTC

Permalink

Hi Stefano!

Post by Stefano Rago
would you mind providing us with the content of your trac.ini file and
installed plugins list?
Make sure you remove any sensitive data from the trac.ini file first.

Contents of trac.ini:

# -*- coding: utf-8 -*-

[agilo-backlogs]
milestone_backlog.columns = status, total_remaining_time, businessvalue,
component, total_story_points, mandatory_story_points
milestone_backlog.name = Milestone Backlog
product_backlog.columns = priority, roi, businessvalue, total_story_points
product_backlog.include_planned_items = True
product_backlog.name = Product Backlog
product_backlog_with_user_stories.columns = priority, roi, businessvalue,
rd_points|total_story_points, sprint, status
product_backlog_with_user_stories.include_planned_items = True
product_backlog_with_user_stories.name = Product Backlog with User Stories
sprint_backlog.charts = burndown
sprint_backlog.columns = rd_points, remaining_time|total_remaining_time,
status, owner
sprint_backlog.include_planned_items = False
sprint_backlog.name = Sprint Backlog
test_backlog.columns = keywords
test_backlog.name = Test Backlog
theme_backlog.columns = priority, roi, businessvalue, total_story_points,
milestone
theme_backlog.include_planned_items = True
theme_backlog.name = Theme Backlog
user_story_backlog.columns = sprint, us_priority, rd_points, status,
keywords
user_story_backlog.include_planned_items = True
user_story_backlog.name = User Story Backlog

[agilo-general]
sprints_can_start_or_end_on_weekends = False

[agilo-links]
allow = theme-requirement, requirement-story, story-task, story-testcase
bug.task.copy = owner, sprint
bug.task.show = owner, remaining_time
cache.related = false
cache.timeout = 0
requirement.calculate =
total_story_points=sum:get_outgoing.rd_points|status=new
requirement.story.copy = owner
requirement.story.show = rd_points, story_priority
story.calculate = total_remaining_time=sum:get_outgoing.remaining_time
story.task.copy = owner, sprint
story.task.show = remaining_time, owner
story.testcase.copy = status, description, resolution, sprint
story.testcase.show = status, testcase_result, resolution, remaining_time,
owner
theme.calculate =
roi=div:businessvalue;total_story_points,total_story_points=sum:get_outgoing.total_story_points
theme.requirement.copy = status, milestone, owner
theme.requirement.show = businessvalue, milestone, rd_points

[agilo-types]
bug = version, sprint, remaining_time
bug.alias = Bug
requirement = component, milestone, keywords, release_note
requirement.alias = Requirement
story = story_priority, keywords, rd_points, sprint, us_priority,
release_note
story.alias = User Story
task = drp_resources, remaining_time, version, sprint, release_note
task.alias = Task
testcase = testcase_result, remaining_time, version, is_writeable_field,
sprint
testcase.alias = Test case
theme = businessvalue, priority, milestone
theme.alias = Theme

[attachment]
max_size = 262144
max_zip_size = 2097152
render_unsafe_content = false

[browser]
color_scale = True
downloadable_paths = /trunk, /branches/*, /tags/*
hide_properties = svk:merge
intermediate_color =
intermediate_point =
newest_color = (255, 136, 136)
oldest_color = (136, 136, 255)
oneliner_properties = trac:summary
render_unsafe_content = false
wiki_properties = trac:description

[changeset]
max_diff_bytes = 10000000
max_diff_files = 0
wiki_format_messages = true

[components]
agilo.* = enabled
agilo.scrum.backlog.web_ui.backlogcontentconverter = enabled
agilo.scrum.workflow.rules.cleanlettersfromremainingtimerule = disabled
agilo_common.* = enabled
agilo_pro.* = enabled
agilo_pro.web_ui.whiteboardnavigationitem = enabled
testmanagementplugin.* = enabled
trac.ticket.api.ticketsystem = disabled
trac.ticket.roadmap.roadmapmodule = disabled
trac.ticket.web_ui.ticketmodule = disabled
tracopt.ticket.commit_updater.committicketreferencemacro = enabled
tracopt.ticket.commit_updater.committicketupdater = enabled
tracopt.versioncontrol.svn.svn_fs.subversionconnector = enabled
tracopt.versioncontrol.svn.svn_prop.subversionmergepropertydiffrenderer =
enabled
tracopt.versioncontrol.svn.svn_prop.subversionmergepropertyrenderer =
enabled
tracopt.versioncontrol.svn.svn_prop.subversionpropertyrenderer = enabled

[header_logo]
alt = XXXX project logo
height = 71
link =
src = site/XXXX_logo.png
width = 255

[inherit]
htdocs_dir =
plugins_dir =
templates_dir =
/usr/local/lib/python2.7/dist-packages/binary_agilo-1.3.10_PRO-py2.7.egg/agilo/templates

[logging]
log_file = /home/trac/trac.log
log_level = DEBUG
log_type = file

[milestone]
stats_provider = DefaultTicketGroupStatsProvider

[mimeviewer]
enscript_modes = text/x-dylan:dylan:4
enscript_path = enscript
max_preview_size = 262144
mime_map = text/x-dylan:dylan, text/x-idl:ice, text/x-ada:ads:adb
php_path = php
pygments_default_style = trac
pygments_modes =
tab_width = 8
treat_as_binary =
application/octet-stream,application/pdf,application/postscript,application/rtf

[notification]
admit_domains =
always_notify_owner = false
always_notify_reporter = false
always_notify_updater = true
ambiguous_char_width = single
batch_subject_template = $prefix Batch modify: $tickets_descr
email_sender = SmtpEmailSender
ignore_domains =
mime_encoding = none
smtp_always_bcc =
smtp_always_cc =
smtp_default_domain =
smtp_enabled = true
smtp_from = xxxx-sL0lkNL+***@public.gmane.org
smtp_from_name =
smtp_password =
smtp_port = 25
smtp_replyto = xxxx-sL0lkNL+***@public.gmane.org
smtp_server = xxxx.fi
smtp_subject_prefix = __default__
smtp_user =
ticket_subject_template = $prefix #$ticket.id: $summary
use_public_cc = false
use_short_addr = false
use_tls = false

[project]
admin =
admin_trac_url = .
descr = My example project
footer = Visit the Trac open source project at<br /><a
href="http://trac.edgewall.org/">http://trac.edgewall.org/</a>
icon = agilo/images/favicon.ico
name = xxxx
url =

[query]
default_anonymous_query = status!=closed&cc~=$USER
default_query = status!=closed&owner=$USER
items_per_page = 100

[report]
items_per_page = 100
items_per_page_rss = 0

[revisionlog]
default_log_limit = 100
graph_colors = ['#cc0', '#0c0', '#0cc', '#00c', '#c0c', '#c00']

[roadmap]
stats_provider = DefaultTicketGroupStatsProvider

[search]
min_query_length = 3

[svn]
branches = trunk,branches/*
tags = tags/*

[testManagementExtension]
subversionpathtotestcases = XXXX/trunk/src/testcases

[ticket]
commit_ticket_update_check_perms = false
commit_ticket_update_commands.close =
commit_ticket_update_commands.refs = <ALL>
commit_ticket_update_envelope =
commit_ticket_update_notify = false
default_cc =
default_component = XXXX
default_description =
default_keywords =
default_milestone = Not set
default_owner =
default_priority = 19
default_resolution = fixed
default_severity =
default_summary =
default_type = defect
default_version =
max_comment_size = 262144
max_description_size = 262144
preserve_newlines = default
restrict_owner = false
workflow = ConfigurableTicketWorkflow

[ticket-custom]
businessvalue = text
businessvalue.label = Business Value
businessvalue.options = |100|200|300|500|800|1200|2000|3000
businessvalue.order = 0
drp_resources = text
drp_resources.label = Resources
i_links = text
i_links.label = Referenced By
is_writeable_field = text
is_writeable_field.label = Is Writeable
is_writeable_field.order = 11
o_links = text
o_links.label = References
rd_points = select
rd_points.label = Story Points
rd_points.options = |0|1|2|3|5|8|13|20|40|100
rd_points.order = 0
release_note = textarea
release_note.cols = 68
release_note.format = wiki
release_note.label = Change Log
release_note.rows = 5
remaining_time = text
remaining_time.label = Remaining Time
sprint = select
sprint.label = Sprint
sprint.value =
story_priority = select
story_priority.label = Importance
story_priority.options = |Mandatory|Linear|Exciter
story_priority.value =
testcase_result = select
testcase_result.label = Test Case Result
testcase_result.options = pass|fail|incomplete
testcase_result.value =
us_priority = text
us_priority.label = Priority
us_priority.order = 9
us_priority.value = 0

[ticket-workflow]
accept = new -> assigned
accept.operations = set_owner_to_self
accept.permissions = TICKET_MODIFY
leave = * -> *
leave.default = 1
leave.operations = leave_status
pass = testing -> closed
pass.name = Passes Testing
pass.operations = set_resolution
pass.set_resolution = fixed
reassign = new,assigned,reopened -> new
reassign.operations = set_owner
reassign.permissions = TICKET_MODIFY
reject = testing -> assigned
reject.name = Failed testing, return to developer
reopen = closed -> reopened
reopen.operations = del_resolution
reopen.permissions = TICKET_CREATE
resolve = new,assigned,reopened -> closed
resolve.operations = set_resolution
resolve.permissions = TICKET_MODIFY
testing = new,assigned -> testing
testing.name = Submit to reporter for testing
testing.permissions = TICKET_MODIFY

[timeline]
abbreviated_messages = True
changeset_collapse_events = false
changeset_long_messages = false
changeset_show_files = 0
default_daysback = 14
max_daysback = 30
newticket_formatter = oneliner
ticket_show_details = false

[trac]
auth_cookie_lifetime = 0
auth_cookie_path =
authz_file =
authz_module_name =
auto_preview_timeout = 2.0
auto_reload = False
backup_dir = db
base_url =
check_auth_ip = false
database = postgres://xxxx:***@localhost:5432/trac
debug_sql = True
default_charset = utf-8
default_dateinfo_format = relative
genshi_cache_size = 128
htdocs_location =
ignore_auth_case = false
jquery_location =
jquery_ui_location =
jquery_ui_theme_location =
mainnav = wiki, timeline, roadmap, browser, tickets, newticket, search
metanav = login, logout, prefs, help, about
mysqldump_path = mysqldump
never_obfuscate_mailto = false
permission_policies = AgiloPolicy, DefaultPermissionPolicy,
LegacyAttachmentPolicy
permission_store = DefaultPermissionStore
pg_dump_path = pg_dump
repository_dir =
repository_sync_per_request = (default)
repository_type = svn
resizable_textareas = true
secure_cookies = False
show_email_addresses = false
show_ip_addresses = false
timeout = 20
use_base_url_for_redirect = False

[versioncontrol]
allowed_repository_dir_prefixes =

[wiki]
ignore_missing_pages = false
max_size = 262144
render_unsafe_content = false
safe_schemes = cvs, file, ftp, git, irc, http, https, news, sftp, smb, ssh,
svn, svn+ssh
split_page_names = false

There are references to SVN, but the debug installation doesn't have access
to version control.
The production system uses post-commit hook to add commits to tickets, but
this doesn't seem to affect performance

Here's how the debug system was installed.
It's a copy/paste from Wiki-page, so might not be the most readable

sudo apt-get install python
sudo apt-get install python-dev
sudo apt-get install libpq-dev
sudo apt-get install python-pip
sudo pip install trac psycopg2

Import the dump. This takes about an hour
psql -a -f /home/trac/postgres_dump_inserts_20130813.sql

sudo trac-admin /var/local/trac/XXXX initenv

Copy old trac.ini and merge changes.
sudo trac-admin /var/local/trac/XXXX upgrade

trac-admin /var/local/trac/XXXX deploy /var/www/trac/XXXX

NOTE running ./trac.fcgi at command line produced an error claiming ctrl+m
characters in the file.
cat trac.fcgi and copy paste to a new file fixed this

sudo easy_install download/binary_agilo-1.3.10_PRO-py2.7.egg

sudo apt-get install libapache2-mod-fcgi

sudo apt-get install libapache2-mod-auth-radius

sudo vi /etc/apache2/apache2.conf

AddRadiusAuth xxxxxxxxxxxxx
AddRadiusCookieValid 5

sudo vi /etc/apache2/httpd.conf

Alias /trac/chrome/common /var/www/trac/XXXX/htdocs/common
Alias /trac/chrome/site /var/www/trac/XXXX/htdocs/site
Alias /trac/chrome/agilo /var/www/trac/XXXX/htdocs/agilo
Alias /trac/chrome/agilo_common /var/www/trac/XXXX/htdocs/agilo_common
Alias /trac/chrome/agilo_pro /var/www/trac/XXXX/htdocs/agilo_pro
<Directory "/var/www/trac/XXXX">
AuthRadiusActive On
Require valid-user
Options +FollowSymLinks
AllowOverride all
Order allow,deny
Allow from all
</Directory>
<Directory "/var/www/trac/XXXX/cgi-bin">
AllowOverride None
Options ExecCGI -MultiViews +FollowSymLinks
AddHandler cgi-script .cgi
Order deny,allow
Allow from all
</Directory>
ScriptAlias /trac /var/www/trac/XXXX/cgi-bin/trac.fcgi

sudo vi /var/www/trac/XXXX/.htaccess

AuthType Basic
AuthName "xxxx"
AuthBasicAuthoritative Off
AuthBasicProvider radius
AuthRadiusAuthoritative on
AuthRadiusActive On
Require valid-user

Post by Stefano Rago
Also, would you be favourable to send us a dump of your database for us to
profile the system with this specific workload?
That is not my decision to make.

I'll see if I can make a smaller dump that still produces the effect.
Since the major time is consumed by looping all team members for each
sprint,
I think the slowness would still be there even with empty sprints and empty
commits.

Disabling Radius authentication didn't affect the performace
If you have other ideas to try out, I can mess freely with the debug trac
installation.

Appreciated,
Torq
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.

Claudio Di Cosmo

2013-10-02 07:46:35 UTC

Permalink

Hi torq,

Post by t***@public.gmane.org
[agilo-links]
cache.timeout = 0

By default, agilo for trac configuration has the ticket caching system disabled. In order to use it it will be necessary to set how much time the cache will be valid every time, changing the cache.tumeout value.
You can start from 2000 (the precision is in milliseconds), see how it goes and then increase it to get the best performance.
If you have still problems, please contact us.

Cheers,

Claudio Di Cosmo
Software Engineer
Agilo Software GmbH
Gruenberger Str. 54
10245 Berlin, Germany

claudio.dicosmo-CKHYNj9aYvRx67MzidHQgQC/***@public.gmane.org
http://www.agilosoftware.com

Follow us on twitter: http://twitter.com/agiloforscrum, http://twitter.com/agilofortrac

Amtsgericht Charlottenburg: HRB 127146
CEO Marion Eickmann, Andrea Tomasini
--
--
Follow Agilo on Twitter: http://twitter.com/agilofortrac
Please support us by reviewing and voting on:
http://userstories.com/products/8-agilo-for-scrum
http://ohloh.net/p/agilo-trac
http://freshmeat.net/projects/agiloforscrum

You have received this message because you are subscribed to
the "Agilo for Trac" Google Group. This group is focused on
supporting Agilo for Trac users and is moderated by
Agilo Software GmbH <http://www.agilosoftware.com>.

To post to this group, send email to agilo-/***@public.gmane.org
To unsubscribe from this group, send an email to
agilo+unsubscribe-/***@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/agilo

---
You received this message because you are subscribed to the Google Groups "Agilo for Trac" group.
To unsubscribe from this group and stop receiving emails from it, send an email to agilo+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
For more options, visit https://groups.google.com/groups/opt_out.