Hello,
Have a SQL 2000 server, and need to make a final decision on my RAID setup
for best perfromance, cost, and drive usage.
MY database application is of a front office/backoffice type, which means
accounting payroll etc is involved as well. Most activity from the front
end are reads, with occ writes. The back office during payroll processing
will generate a lot of writes in a short period of time. I only have 50
users using this application.
I have A single processor dell server with a 1x6 backplane with 6 hot swap
bays. I currently have 3-36 GB drives to use.
I have the option to split the backplane. My RAID choices are as follows:
1x6 backplane (I think all I/O's go through 1 channel/port/cable on
controller to all 6 drives in this 1x6 backplane setup)
2 - Raid 5 arrays (3 drives each)
OR
RAID 1 (2 HD's) and a RAID 10 (4 HD's) array (all through single
port/channel/cable)
IF I split the backplane to a 2x3 backplane (uses 2 channels/ports/cables on
the controller)
2 - RAID 5 arrays (3 HD's each)
OR
1- RAID 1 array & 1- RAID 5 array on separate channels/ports (This leaves 1
bay open and disable my ability to use RAID 10)
I have the OS, SQL exe's, SQL databases, Trancaction logs and tempdb logs,
and backups to deal with.
I thought I read somewhere if the transaction log files are not on a RAID 1
volume by themselves, then it defeats the purpose. But I'm unsure if they
meant separate physical drives, or just separate volumes or partitions on the
same drives.
So my issues are I/O's from controller channels/ports, separate physical
RAID volumes, and then partitions (or volumes) on the RAID drives.
Based on this information, can anyone reflect from experience which of my
choices above would work out the best for my SQL application?
Thanks much,
MarkHi
With the number of drives you have available then you are going to have to
compromise in some way. Ideally you would separate OS, Pagefile, tempdb,
system databases, user database data files, user database log file all onto
separate sets of spindles or possibly multiple sets of spindles if you have
multiple files for each. Where you compromise may be dependent on the
application usage and things like the amount of memory installed/used. I
would probably go for the Raid 1/Raid 10 configuration with data and log
files together, but monitor how the system copes and see about moving things
around to get the best you can.
You may want to look at
http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/sqlops6.mspx
and
http://www.microsoft.com/technet/prodtechnol/sql/2005/physdbstor.mspx
John
"Mrpush" wrote:
> Hello,
> Have a SQL 2000 server, and need to make a final decision on my RAID setup
> for best perfromance, cost, and drive usage.
> MY database application is of a front office/backoffice type, which means
> accounting payroll etc is involved as well. Most activity from the front
> end are reads, with occ writes. The back office during payroll processing
> will generate a lot of writes in a short period of time. I only have 50
> users using this application.
> I have A single processor dell server with a 1x6 backplane with 6 hot swap
> bays. I currently have 3-36 GB drives to use.
> I have the option to split the backplane. My RAID choices are as follows:
> 1x6 backplane (I think all I/O's go through 1 channel/port/cable on
> controller to all 6 drives in this 1x6 backplane setup)
> 2 - Raid 5 arrays (3 drives each)
> OR
> RAID 1 (2 HD's) and a RAID 10 (4 HD's) array (all through single
> port/channel/cable)
>
> IF I split the backplane to a 2x3 backplane (uses 2 channels/ports/cables on
> the controller)
> 2 - RAID 5 arrays (3 HD's each)
> OR
> 1- RAID 1 array & 1- RAID 5 array on separate channels/ports (This leaves 1
> bay open and disable my ability to use RAID 10)
> I have the OS, SQL exe's, SQL databases, Trancaction logs and tempdb logs,
> and backups to deal with.
> I thought I read somewhere if the transaction log files are not on a RAID 1
> volume by themselves, then it defeats the purpose. But I'm unsure if they
> meant separate physical drives, or just separate volumes or partitions on the
> same drives.
> So my issues are I/O's from controller channels/ports, separate physical
> RAID volumes, and then partitions (or volumes) on the RAID drives.
> Based on this information, can anyone reflect from experience which of my
> choices above would work out the best for my SQL application?
> Thanks much,
> Mark
>
>
>|||John,
Thanks for the info.
Just to confirm, you believe I should go with the Raid 1, with OS and SQL
exe's on it.
Then I'd have a Raid 10 with all SQL database and log files (Master, temp, &
User databases and log files together)
Where best do my backup filea go? The Raid 1 or 10 Array? They will be
done nightly, when nearly no DB's are being used.)
Also, are there any free utilities I could use to load test the system and
track the I/O loads with the different RAID configs?
Thanks much,
Mark
"John Bell" wrote:
> Hi
> With the number of drives you have available then you are going to have to
> compromise in some way. Ideally you would separate OS, Pagefile, tempdb,
> system databases, user database data files, user database log file all onto
> separate sets of spindles or possibly multiple sets of spindles if you have
> multiple files for each. Where you compromise may be dependent on the
> application usage and things like the amount of memory installed/used. I
> would probably go for the Raid 1/Raid 10 configuration with data and log
> files together, but monitor how the system copes and see about moving things
> around to get the best you can.
> You may want to look at
> http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/sqlops6.mspx
> and
> http://www.microsoft.com/technet/prodtechnol/sql/2005/physdbstor.mspx
> John
> "Mrpush" wrote:
> > Hello,
> >
> > Have a SQL 2000 server, and need to make a final decision on my RAID setup
> > for best perfromance, cost, and drive usage.
> >
> > MY database application is of a front office/backoffice type, which means
> > accounting payroll etc is involved as well. Most activity from the front
> > end are reads, with occ writes. The back office during payroll processing
> > will generate a lot of writes in a short period of time. I only have 50
> > users using this application.
> >
> > I have A single processor dell server with a 1x6 backplane with 6 hot swap
> > bays. I currently have 3-36 GB drives to use.
> >
> > I have the option to split the backplane. My RAID choices are as follows:
> >
> > 1x6 backplane (I think all I/O's go through 1 channel/port/cable on
> > controller to all 6 drives in this 1x6 backplane setup)
> >
> > 2 - Raid 5 arrays (3 drives each)
> > OR
> > RAID 1 (2 HD's) and a RAID 10 (4 HD's) array (all through single
> > port/channel/cable)
> >
> >
> > IF I split the backplane to a 2x3 backplane (uses 2 channels/ports/cables on
> > the controller)
> >
> > 2 - RAID 5 arrays (3 HD's each)
> > OR
> > 1- RAID 1 array & 1- RAID 5 array on separate channels/ports (This leaves 1
> > bay open and disable my ability to use RAID 10)
> >
> > I have the OS, SQL exe's, SQL databases, Trancaction logs and tempdb logs,
> > and backups to deal with.
> >
> > I thought I read somewhere if the transaction log files are not on a RAID 1
> > volume by themselves, then it defeats the purpose. But I'm unsure if they
> > meant separate physical drives, or just separate volumes or partitions on the
> > same drives.
> >
> > So my issues are I/O's from controller channels/ports, separate physical
> > RAID volumes, and then partitions (or volumes) on the RAID drives.
> >
> > Based on this information, can anyone reflect from experience which of my
> > choices above would work out the best for my SQL application?
> >
> > Thanks much,
> >
> > Mark
> >
> >
> >
> >
> >|||Hi Mark
You could back up to the RAID 1 discs or possibly a network (UNC) drive. You
will have the option of moving either the log file from RAID 10 discs to the
RAID 1 disc and benchmarking the performance. If you have significant use of
the page file, you may want to see about adding more memory.
John
"Mrpush" wrote:
> John,
> Thanks for the info.
> Just to confirm, you believe I should go with the Raid 1, with OS and SQL
> exe's on it.
> Then I'd have a Raid 10 with all SQL database and log files (Master, temp, &
> User databases and log files together)
> Where best do my backup filea go? The Raid 1 or 10 Array? They will be
> done nightly, when nearly no DB's are being used.)
> Also, are there any free utilities I could use to load test the system and
> track the I/O loads with the different RAID configs?
> Thanks much,
> Mark
>
>
>
> "John Bell" wrote:
> > Hi
> >
> > With the number of drives you have available then you are going to have to
> > compromise in some way. Ideally you would separate OS, Pagefile, tempdb,
> > system databases, user database data files, user database log file all onto
> > separate sets of spindles or possibly multiple sets of spindles if you have
> > multiple files for each. Where you compromise may be dependent on the
> > application usage and things like the amount of memory installed/used. I
> > would probably go for the Raid 1/Raid 10 configuration with data and log
> > files together, but monitor how the system copes and see about moving things
> > around to get the best you can.
> >
> > You may want to look at
> > http://www.microsoft.com/technet/prodtechnol/sql/2000/maintain/sqlops6.mspx
> >
> > and
> >
> > http://www.microsoft.com/technet/prodtechnol/sql/2005/physdbstor.mspx
> >
> > John
> >
> > "Mrpush" wrote:
> >
> > > Hello,
> > >
> > > Have a SQL 2000 server, and need to make a final decision on my RAID setup
> > > for best perfromance, cost, and drive usage.
> > >
> > > MY database application is of a front office/backoffice type, which means
> > > accounting payroll etc is involved as well. Most activity from the front
> > > end are reads, with occ writes. The back office during payroll processing
> > > will generate a lot of writes in a short period of time. I only have 50
> > > users using this application.
> > >
> > > I have A single processor dell server with a 1x6 backplane with 6 hot swap
> > > bays. I currently have 3-36 GB drives to use.
> > >
> > > I have the option to split the backplane. My RAID choices are as follows:
> > >
> > > 1x6 backplane (I think all I/O's go through 1 channel/port/cable on
> > > controller to all 6 drives in this 1x6 backplane setup)
> > >
> > > 2 - Raid 5 arrays (3 drives each)
> > > OR
> > > RAID 1 (2 HD's) and a RAID 10 (4 HD's) array (all through single
> > > port/channel/cable)
> > >
> > >
> > > IF I split the backplane to a 2x3 backplane (uses 2 channels/ports/cables on
> > > the controller)
> > >
> > > 2 - RAID 5 arrays (3 HD's each)
> > > OR
> > > 1- RAID 1 array & 1- RAID 5 array on separate channels/ports (This leaves 1
> > > bay open and disable my ability to use RAID 10)
> > >
> > > I have the OS, SQL exe's, SQL databases, Trancaction logs and tempdb logs,
> > > and backups to deal with.
> > >
> > > I thought I read somewhere if the transaction log files are not on a RAID 1
> > > volume by themselves, then it defeats the purpose. But I'm unsure if they
> > > meant separate physical drives, or just separate volumes or partitions on the
> > > same drives.
> > >
> > > So my issues are I/O's from controller channels/ports, separate physical
> > > RAID volumes, and then partitions (or volumes) on the RAID drives.
> > >
> > > Based on this information, can anyone reflect from experience which of my
> > > choices above would work out the best for my SQL application?
> > >
> > > Thanks much,
> > >
> > > Mark
> > >
> > >
> > >
> > >
> > >|||Mrpush wrote:
> John,
> Thanks for the info.
> Just to confirm, you believe I should go with the Raid 1, with OS and SQL
> exe's on it.
> Then I'd have a Raid 10 with all SQL database and log files (Master, temp, &
> User databases and log files together)
> Where best do my backup filea go? The Raid 1 or 10 Array? They will be
> done nightly, when nearly no DB's are being used.)
> Also, are there any free utilities I could use to load test the system and
> track the I/O loads with the different RAID configs?
> Thanks much,
> Mark
>
>
Hi Mark
I'd rather go for a solution where you have your log and data files
separated. This is both from a performance and disaster recovery point
of view. If you have both log and data files on the same array/disks,
then if this array fails and you loose you files you've lost everything.
From a performance point of view it will in most case give better
performance to split log and data files to seperate arrays/drives and
best of all different RAID controller as well. For disaster recovery
reasons, I'd also try to put my backup files on a array that doesn't
contains any data or log files.
With your limited server configuration, I know it's limited how much you
can do, but I think I'd go for a RAID1 for your log files and a RAID 5
for your data files. If you don't do log backups though (if your
databases are in SIMPLE recovery), you could from a disaster recovery
point of view put both log and data files on the same array and then put
your backup on a seperate array. If you log/data array then fails and
you loose your files, it doesn't really matter that you loose both log
and data files since your only option will be to restore the last full
backup any way.
Regards
Steen Schlüter Persson
Databaseadministrator / Systemadministrator|||Hi
RAID10 will have better performance on writes and more better fault tolerance.
John
"Steen Persson (DK)" wrote:
> Mrpush wrote:
> > John,
> >
> > Thanks for the info.
> >
> > Just to confirm, you believe I should go with the Raid 1, with OS and SQL
> > exe's on it.
> >
> > Then I'd have a Raid 10 with all SQL database and log files (Master, temp, &
> > User databases and log files together)
> >
> > Where best do my backup filea go? The Raid 1 or 10 Array? They will be
> > done nightly, when nearly no DB's are being used.)
> >
> > Also, are there any free utilities I could use to load test the system and
> > track the I/O loads with the different RAID configs?
> >
> > Thanks much,
> >
> > Mark
> >
> >
> >
> Hi Mark
> I'd rather go for a solution where you have your log and data files
> separated. This is both from a performance and disaster recovery point
> of view. If you have both log and data files on the same array/disks,
> then if this array fails and you loose you files you've lost everything.
> From a performance point of view it will in most case give better
> performance to split log and data files to seperate arrays/drives and
> best of all different RAID controller as well. For disaster recovery
> reasons, I'd also try to put my backup files on a array that doesn't
> contains any data or log files.
> With your limited server configuration, I know it's limited how much you
> can do, but I think I'd go for a RAID1 for your log files and a RAID 5
> for your data files. If you don't do log backups though (if your
> databases are in SIMPLE recovery), you could from a disaster recovery
> point of view put both log and data files on the same array and then put
> your backup on a seperate array. If you log/data array then fails and
> you loose your files, it doesn't really matter that you loose both log
> and data files since your only option will be to restore the last full
> backup any way.
>
> --
> Regards
> Steen Schlüter Persson
> Databaseadministrator / Systemadministrator
>|||John Bell wrote:
> Hi
> RAID10 will have better performance on writes and more better fault tolerance.
> John
>
Agreed, but with only 6 drives available it might be difficult to use 4
of them for just one RAID10 array. Since the OP indicates that it's
mostly reads that are performed it might be better to create a RAID 5
array for the database file.
Regards
Steen Schlüter Persson
Databaseadministrator / Systemadministrator|||But then... if it is mainly reads the location of the log file is less of a
factor!!! :)
Unfortunately there is no absolute answer to this question, and too many
variables are still outstanding, Mark (the OP) would need to do some
benchmarking and load testing to find what is best for himself!!
John
"Steen Persson (DK)" wrote:
> John Bell wrote:
> > Hi
> >
> > RAID10 will have better performance on writes and more better fault tolerance.
> >
> > John
> >
> >
> Agreed, but with only 6 drives available it might be difficult to use 4
> of them for just one RAID10 array. Since the OP indicates that it's
> mostly reads that are performed it might be better to create a RAID 5
> array for the database file.
>
> --
> Regards
> Steen Schlüter Persson
> Databaseadministrator / Systemadministrator
>|||Guys,
Thanks for all the input.
In all actuallity, my take on the reads is an assumption really. I naeer
have bench marked it as its a new system but the general nature of the app is
more of just pulling data to view in 75% of the cases. The back end will
have much more writes however during payroll processing.
If I do the Raid 1/10, I'd have to purchase ~$1000 worth of drives. This
would give me 36GB for the raid 1 and 146GB for the Raid 10 array.
(in comparison, the Raid 1/5 would cost ~800 and give me an extra 70 GB
space).
The DB's are not huge, approx 2GB that will probably only grow ~.5 GB per
year.
Question, are there some free/cheap tools I can use to benchmark or load
test the server with reads and writes?
Thanks much,
Mark
"John Bell" wrote:
> But then... if it is mainly reads the location of the log file is less of a
> factor!!! :)
> Unfortunately there is no absolute answer to this question, and too many
> variables are still outstanding, Mark (the OP) would need to do some
> benchmarking and load testing to find what is best for himself!!
> John
> "Steen Persson (DK)" wrote:
> > John Bell wrote:
> > > Hi
> > >
> > > RAID10 will have better performance on writes and more better fault tolerance.
> > >
> > > John
> > >
> > >
> >
> > Agreed, but with only 6 drives available it might be difficult to use 4
> > of them for just one RAID10 array. Since the OP indicates that it's
> > mostly reads that are performed it might be better to create a RAID 5
> > array for the database file.
> >
> >
> > --
> > Regards
> > Steen Schlüter Persson
> > Databaseadministrator / Systemadministrator
> >|||Hi Mark
You can replay a trace using SQL profiler, this will depend on the original
workload and may not really be the best way of stress testing the disc
subsystem. Similarly if your applications are web based then you could
possibly use ACT http://tinyurl.com/lohup.
SQLIOStress is a free tool http://support.microsoft.com/kb/231619 that will
stress your disc subsystem, and there is also SQLIO http://tinyurl.com/fyt53.
Some of the other products you may want to look at are LoadRunner
http://www.mercury.com/us/products/performance-center/loadrunner/
or IBM Rational performance tester
http://www-306.ibm.com/software/awdtools/tester/performance/index.html or
Benchmark factory from Quest
http://www.quest.com/benchmark_factory/default.aspx. Y
John
"Mrpush" wrote:
> Guys,
> Thanks for all the input.
> In all actuallity, my take on the reads is an assumption really. I naeer
> have bench marked it as its a new system but the general nature of the app is
> more of just pulling data to view in 75% of the cases. The back end will
> have much more writes however during payroll processing.
> If I do the Raid 1/10, I'd have to purchase ~$1000 worth of drives. This
> would give me 36GB for the raid 1 and 146GB for the Raid 10 array.
> (in comparison, the Raid 1/5 would cost ~800 and give me an extra 70 GB
> space).
> The DB's are not huge, approx 2GB that will probably only grow ~.5 GB per
> year.
> Question, are there some free/cheap tools I can use to benchmark or load
> test the server with reads and writes?
> Thanks much,
> Mark
>
>
> "John Bell" wrote:
> > But then... if it is mainly reads the location of the log file is less of a
> > factor!!! :)
> >
> > Unfortunately there is no absolute answer to this question, and too many
> > variables are still outstanding, Mark (the OP) would need to do some
> > benchmarking and load testing to find what is best for himself!!
> >
> > John
> >
> > "Steen Persson (DK)" wrote:
> >
> > > John Bell wrote:
> > > > Hi
> > > >
> > > > RAID10 will have better performance on writes and more better fault tolerance.
> > > >
> > > > John
> > > >
> > > >
> > >
> > > Agreed, but with only 6 drives available it might be difficult to use 4
> > > of them for just one RAID10 array. Since the OP indicates that it's
> > > mostly reads that are performed it might be better to create a RAID 5
> > > array for the database file.
> > >
> > >
> > > --
> > > Regards
> > > Steen Schlüter Persson
> > > Databaseadministrator / Systemadministrator
> > >|||This is a multi-part message in MIME format.
--020401070701080401090805
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Mrpush wrote:
> Guys,
> Thanks for all the input.
> In all actuallity, my take on the reads is an assumption really. I naeer
> have bench marked it as its a new system but the general nature of the app is
> more of just pulling data to view in 75% of the cases. The back end will
> have much more writes however during payroll processing.
> If I do the Raid 1/10, I'd have to purchase ~$1000 worth of drives. This
> would give me 36GB for the raid 1 and 146GB for the Raid 10 array.
> (in comparison, the Raid 1/5 would cost ~800 and give me an extra 70 GB
> space).
> The DB's are not huge, approx 2GB that will probably only grow ~.5 GB per
> year.
> Question, are there some free/cheap tools I can use to benchmark or load
> test the server with reads and writes?
> Thanks much,
> Mark
>
>
Hi Mark
Doing good performance tests are a bit difficult since it's hard to get
exactly the same load/working pattern that you have in real life
production. Some of the test tools and ideas John has suggested will get
you started though.
We have just bought a new SAN (IBM DS4300 with 3 extra bays which means
56 disks in total) and I've had the change to test this in various
configurations. From previous performance monitoring on our existing
server/SAN, we know that we have a high number of reads. I think it's
something like 95% READS and 5 % writes - or maybe even bigger difference.
I've only focused on the data file because the log file will be on a
RAID1/10 but I've tested the performance with the data file on RAID 5
and RAID 1/10 array.
I'm not done with the testing and the conclusion yet, but my first
impression is that RAID5 still gives the best read performance (that was
also what I expected). I've set a maximum of 28 disks that I can use for
my data array and I've tested with 28 disks in RAID 5, 28 disks in
RAID1/10 and with 14 disks in RAID5. Even with only 14 disks in RAID5, I
got a slightly better read performance than with RAID1/10.
The write performance is of course better on RAID1/10, but actually the
difference seems smaller than I'd have expected.
It's not a very "scientific" test I've done, so there can be a lot of
factors that I haven't countered for. Also the purpose of the test
wasn't to see a generel difference between RAID5 and RAID1/10, but only
to see the difference on this specific SAN with it's specific
configuration. This means that there can very well be controller
settings, cache settings etc. that "masks" the result and maybe levels
out the difference between the various RAID configurations. In the IBM
storage manager, they also treats RAID1 and RAID10 as the same so I'm
not quite sure that it creates a true RAID10 array. I've talked to IBM
about this, but I'm still waiting on a reply on this (the guy I should
talk to is on holiday). This little "twist" can of course also make a
difference in the performance of the different RAIDs.
Regards
Steen Schlüter Persson
Databaseadministrator / Systemadministrator
--020401070701080401090805
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Mrpush wrote:
<blockquote cite="mid4080BB3E-3E98-4785-A67A-5C5F4357A062@.microsoft.com"
type="cite">
<pre wrap="">Guys,
Thanks for all the input.
In all actuallity, my take on the reads is an assumption really. I naeer
have bench marked it as its a new system but the general nature of the app is
more of just pulling data to view in 75% of the cases. The back end will
have much more writes however during payroll processing.
If I do the Raid 1/10, I'd have to purchase ~$1000 worth of drives. This
would give me 36GB for the raid 1 and 146GB for the Raid 10 array.
(in comparison, the Raid 1/5 would cost ~800 and give me an extra 70 GB
space).
The DB's are not huge, approx 2GB that will probably only grow ~.5 GB per
year.
Question, are there some free/cheap tools I can use to benchmark or load
test the server with reads and writes?
Thanks much,
Mark
</pre>
</blockquote>
<small><font face="Arial">Hi Mark<br>
<br>
Doing good performance tests are a bit difficult since it's hard to get
exactly the same load/working pattern that you have in real life
production. Some of the test tools and ideas John has suggested will
get you started though.<br>
We have just bought a new SAN (IBM DS4300 with 3 extra bays which means
56 disks in total) and I've had the change to test this in various
configurations. From previous performance monitoring on our existing
server/SAN, we know that we have a high number of reads. I think it's
something like 95% READS and 5 % writes - or maybe even bigger
difference. <br>
I've only focused on the data file because the log file will be on a
RAID1/10 but I've tested the performance with the data file on RAID 5
and RAID 1/10 array. <br>
<br>
I'm not done with the testing and the conclusion yet, but my first
impression is that RAID5 still gives the best read performance (that
was also what I expected). I've set a maximum of 28 disks that I can
use for my data array and I've tested with 28 disks in RAID 5, 28 disks
in RAID1/10 and with 14 disks in RAID5. Even with only 14 disks in
RAID5, I got a slightly better read performance than with RAID1/10.<br>
The write performance is of course better on RAID1/10, but actually the
difference seems smaller than I'd have expected.<br>
<br>
It's not a very "scientific" test I've done, so there can be a lot of
factors that I haven't countered for. Also the purpose of the test
wasn't to see a generel difference between RAID5 and RAID1/10, but only
to see the difference on this specific SAN with it's specific
configuration. This means that there can very well be controller
settings, cache settings etc. that "masks" the result and maybe levels
out the difference between the various RAID configurations. In the IBM
storage manager, they also treats RAID1 and RAID10 as the same so I'm
not quite sure that it creates a true RAID10 array. I've talked to IBM
about this, but I'm still waiting on a reply on this (the guy I should
talk to is on holiday). This little "twist" can of course also make a
difference in the performance of the different RAIDs.<br>
<br>
<br>
-- <br>
Regards<br>
Steen Schlüter Persson<br>
Databaseadministrator / Systemadministrator<br>
</font></small>
</body>
</html>
--020401070701080401090805--
No comments:
Post a Comment