Académique Documents
Professionnel Documents
Culture Documents
directory
Option A. Upgrade your database using the Trend Micro DB Upgrade tool (Recommended)
................................................................................................................................................... 2
Known Issue......................................................................................................................... 22
Option B. Manually upgrade your database using the Trend Micro DB Upgrade script ...... 22
Troubleshooting procedure................................................................................................. 36
Notes ................................................................................................................................... 38
Option A. Upgrade your database using the Trend Micro DB Upgrade tool (Recommended)
Download the Trend Micro DB Upgrade tool. This tool mainly consists of a .Net executable binary and specifically designed
SQL scripts.
1. Unzip the TrendDBUpgrade.zip file in a temp directory, preferably on your system drive. The zip file contains the
following:
There might be some little discrepancies between the screenshots shown in this article and the version of
the product you are working with.
The following are the MD5 and SHA 256 value of the TrendDBMigrator.exe file:
MD5 value:
7C9A35AB6B40B9841CBEB3A9C2DB3171
SHA 256
973EFA5C59D214F80BF57C6883196863F681BF3C60FA81F43344AB995D690498
3. The installer will check for dependencies. If the dependencies are not met, it will display the following message:
Microsoft SQL Server 2008 R2 Command Line (SQLCMD) could not be located. Program can not continue
without required dependencies.
Would you like this tool to download and install them for Microsoft Download Center now?
4. The tool has three (3) dependencies that you should install:
c. If your server is not equipped with Microsoft .Net Framework 3.5+, click the Server Manager icon on
Taskbar and select Add roles and features.
d. Walk through the wizard and select .NET framework 3.5 Feature.
g. While downloading the Microsoft SQL, the tool creates a folder named InstallMe under the same extracted
folder from where you have run the installer TrendDBMigrator.exe. The newly created folder contains these
two (2) executable files and their respective log files:
SqlCmdLnUtils
Sqlncli
h. When the download is finished, a confirmation message appears. Click OK to start the tool.
i. You may check the installed dependencies under Control Panel > Program and Features.
5. If the download fails or you choose No, you will see the following message:
Please download and install Microsoft SQL Server 2008 R2 Command Line Utilities (SQLCMD), from
Microsoft download center.
When you click OK, a page corresponding to the missing dependencies will open in a browser, where you
canmanually download and install the dependencies before running the tool.
Once you have installed the tool, you can now utilize it using these three (3) tabs:
Database Settings
Real-time Log
Database Settings
Database Settings is the main tab where you must enter the required information and credentials to connect to your SQL
Server. Once you've entered the SQL Server IP, User Name, and Password (Instance is optional and normally left empty), you
can click the Connect button. If the information is correct, you should see a list of all T0 databases present on that SQL
Server.
All the data in the grid is sortable. To sort the data in the grid, click the appropriate column header. The data will sort in either
ascending or descending order based on the previous state.
If you want to connect to the database using a domain account, enable the Integrated Security option. You can see the
location of this option in the screenshot above.
The image above shows a list of T0 databases. In the Migration Status column, there are results showing"Already
Migrated" or "Not Migrated". The status "Already Migrated" indicates that the database has all the required fields converted to
BIGINT or it is the latest installation and does not need to be migrated. Databases that are already migrated are highlighted in
green.
For databases that are "Not Migrated", you can click them one by one and migrate them. You have two (2) options to migrate
a database:
If you switch between the two Migrating T0 to options, click the Connect button again to fetch the associated results for that
version.
If the Show Multi-tenants DSM Database Only box is checked, you will see only T0 databases that are multi-tenant. If this
box is unchecked, you will see all Deep Security databases present on the SQL Server.
All red buttons are for schema changes. Green buttons cannot change the schema. You can run simulations by clicking Run
Simulation – Selected Databases. Also, you can check space using theCheck Space – Selected Databases button.
Below is a sample scenario wherein the image shows a database named "ak-test2" is selected. Its two (2) tenant databases
(Tn), named "ak-test2_1" and "ak-test2_2", are displayed. Please note the Migration Status of both tenant databases
is "Cannot Migrate". The reason is that the T0 database (ak-test2) has not been migrated yet. This tool does not migrate
tenants together with a T0. It is essential to migrate the T0 first and then migrate all related tenants in one step.
If the tenant appears gray in the grid, it means its database schema has already been migrated to Bigint.
1. Click the database name from T0 and do not select any Tn.
3. Note that this tool calculates the space available before migrating the database. If there is no enough space, it will
display the following warning and close:
Required space is NOT available for migration. It is recommended to allocate more space before migration.
The image above indicates that the available space is 0.2890625 MB. This is not the Disk Drive space calculation.
You can calculate the available space by right-clicking the database in SQL Server and navigating
to Properties > General > Database > Space Available. If the space does not reach at least 150% of the space
currently used by the database, the said warning will appear.
b. Select Files.
c. Make the changes in the Initial Size column to make it more than 150% of the current space and click OK.
In the sample below, the size has been changed to 2048 MB to accommodate 150%, which is
544.74609375 MB.
4. After fixing the available space, click again the Migrate-Selected Database button in the tool. It should now show
that the space is enough to migrate, then click OK.
Once the migration is completed, you will see the log file. The log file name indicates information like the operation
executed, date, time, and database on which that operation was executed.
This log file is placed under the extracted TrenDBUpgrade – Log folder.
At the end of the log file, you will see that the schema migration is completed and the script executed successfully.
8. Go back to tool and you will observe that the Migration Status column for "ak-test2" database is now set to"Already
Migrated".
The tool needs exclusive access to database, so it kills all active sessions and connections, puts the database in
single user mode, executes, and then changes the database back to multi-user again.
The tool also verifies that the credentials used are correct and the user used to connect to the database has the
db_owner role. If not, the credentials are rejected. If the tool produces an error message about wrong credentials
for the selected TN, you can edit the username and password field by clicking the respective cell.
Tenant's Migration
Now it’s time to migrate the Tenant databases. The "ak-test2" database in our example has two (2) tenant databases. You can
migrate a maximum of 10 tenants at a time.
3. The tool will check the available space for both tenants and if there is enough space, it will display the following
information, which is repeated for each tenant that you selected for migration. Click OKto dismiss each message.
4. The amount of time that a tenant migration takes depends on how many databases you have selected. When the
migration is completed, a log file is opened for each tenant, showing that migration has been finished. You can
verify the completed migration in the sample log files below:
5. You can check all the log files saved in the same TrendDBUpgrade – Log folder. You can also verify in the tool
under the Selected T0's Tenants area that each tenant is displayed as "Already Migrated".
6. Go back to the Deep Security Manager and try to run the database upgrade. The installer will now upgrade the
Deep Security Manager smoothly.
3. Press the TAB key and type the password in the User Password column.
As the name indicates, if you do not want to migrate manually, you can run the migration process as a scheduled batch job.
Once you are connected to the SQL Server and have populated the database lists on the Database Settingstab, the same
tenants list will appear on the Scheduled Migration Jobs tab.
Schedule Migration Jobs are only available for tenants and are not applicable for T0. You need to migrate T0 using the
Database Settings tab, as described in previous section.
To schedule migration jobs:
1. Select one or more tenants (maximum of 10 tenants) using the Select check boxes in the data grid.
2. After selecting the required tenants, click Add to Scheduled Job. If the selected database is not already migrated,
it will be added to the Scheduled Jobs list.
3. Click its Toggle button under the Toggle Schedule column to enable a job for execution.
4. Once you toggle the job, a new dialog will appear where you can select a date and time when you want to run the
job. Set the appropriate time and click OK.
5. You can verify the toggled job under the Scheduled column showing a "Yes"status. The Execution Date and Time
are also shown.
To remove the schedule from the Scheduled Job, click Toggle and the Scheduled column will display "No".
To remove a job from the list of Scheduled Jobs, select the job and click Remove Selected Job. If you want to remove all the
jobs, click Remove all Jobs and it will remove all jobs from the list.
Once the tool is executing a job, it cannot be removed from the list. Once the job is scheduled, you cannot exit the tool. It must
be kept running. Logging out of the machine will kill the tool and any scheduled jobs will be lost. If you kill the tool, you have to
recreate and reschedule all the jobs again.
During execution, a scheduled job will produce a log file in theTrendDBUpgrade – Log folder.
Messages related to scheduled jobs do not appear in the Real-time log. When a job is done, it is removed from the Scheduled
Jobs section and the Migration status under T0 Tenants List is changed to "Migrated".
If you scheduled a job under the Schedule Migration Job tab, it is scheduled to be run at a certain time. If you go back to
the Database Settings tab and click Disconnect, it will disconnect the connection from the SQL Server. However, the batch
job will still remain under the Scheduled Migration tab and the job will run at its scheduled time. You will not be able to close
the tool and if you attempt to do so, a message will be displayed, saying that the batch job is scheduled. If you remove the
scheduled job, it will close the tool.
This tab shows messages thrown by various operations being executed. The same messages are saved in their respective log
files as well.
Other Important Notes
In scenarios where the database size is hugs (in several GBs) it may take hours for tool to migrate the database and it
requires patience. It is strongly suggested to run this tool residing on MS SQL Server itself to avoid any network interruption
which may cause problems in upgrading large database schema.
The extracted folder TrendDBUpgrade has these subfolders:
SQL_1
SQL_2
SQL_3
The subfolders contain the following SQL Script files, which should not be modified, deleted, or moved outside this folder.
DeepSecurityDatabaseBigintMigrationScriptSimulation.sql
DeepSecurityDatabaseBigintMigrationScriptTableSpaceUsageSummary.sql
DeepSecurityDatabaseBigintMigrationScriptTableTruncation.sql
DeepSecurityDatabaseBigintMigrationScript.sql
UnlockDatabase.sql
The tool will not run if any of the files mentioned above are missing. You MUST NOT change these files. If you do so, you may
lock or corrupt your database.
This tool, on execution of any operation (other than Usage Summary), sets the Remote Query Timeout value to "0" (which
means no timeout). The default value is 600 seconds. It reverts back this value at the end of the script execution. If the script
fails for any reason, you can find the original value from the log file and manually revert it.
Tool Migration Logic
3. Renames database and append it to "_locked". For example if database name is DSM, then it will be renamed as
"DSM_locked".
5. Run migration /other scripts on renamed database (usage check does not rename database).
If the migration script fails for any reason, it may leave a renamed database or leave the database in single user mode. To fix
this, perform the following steps:
1. Rename the database manually back to its original name by right-clicking the database name and clicking
Rename.
2. Change the Restrict Access to MULTI_USR by right-clicking the database name, clicking Properties, clicking
Options, under State – Restrict Access.
Settings
Connection and Command timeout can be changed using these options. Please do not change the DSM Constraints Count
without consulting SEG engineers.
Known Issue
During testing, it was observed that if customers are upgrading from Deep Security Managers with previously installed
versions from 7.5 or 8.0 to 9.6 SP1 onwards, and that the DSM is a Multi-Tenant Environment, the installer during upgrade
may see a hiccup while accessing the Database Schema and throws an error “Unable to Access Database Schema”.
Click OK to continue.
Log onto the DSM console, and then click Perform Database Upgrade by clicking each tenant.
Option B. Manually upgrade your database using the Trend Micro DB Upgrade script
Download the Trend Micro DB Upgrade script. Updating the database using this script lets you visualize the technical details
during the process.
5. Click Execute button on the SQL Server. The result should be similar to the following:
6. ---------------------------------------------------------------------------
7. @@@ Deep Security Database Migration Script >>> Recovery Commands Preparing
8. ---------------------------------------------------------------------------
9.
11.
13.
15.
17.
29.
31.
32. drop index antimalwareeventhistory_antimalwareeventid on antimalwareeventhistory;
41.
43.
61.
63.
81.
83.
84. alter table integrityeventhistory with check add constraint FDHWRNDGVSXGZGEN foreign
key(integrityeventid) references integrityevents(integrityeventid) on delete cascade;
85. alter table antimalwareeventhistory with check add constraint FEPXAYYXJTHEXHHY foreign
key(antimalwareeventid) references antimalwareevents(antimalwareeventid) on delete cascade;
86. alter table payloadloghistory with check add constraint FFOIMMXCPSVLEINU foreign
key(payloadlogid) references payloadlogs(payloadlogid) on delete cascade;
87. alter table packetloghistory with check add constraint FHJISFEARDLHSTVM foreign
key(packetlogid) references packetlogs(packetlogid) on delete cascade;
88. alter table webreputationeventhistory with check add constraint FLAYJBRFYRTWJNQK foreign
key(webreputationeventid) references webreputationevents(webreputationeventid) on delete
cascade;
89. alter table agentevents with check add constraint FRWQQDGGMZFNONZC foreign key(systemeventid)
references systemevents(systemeventid) on delete cascade;
90. alter table activehosterrors with check add constraint FSMKZGIFLEQVANFB foreign
key(endsystemeventid) references systemevents(systemeventid);
91. alter table loginspectioneventhistory with check add constraint FWLRPXPOEPNVBAII foreign
key(loginspectioneventid) references loginspectionevents(loginspectioneventid) on delete
cascade;
92. alter table systemeventhistory with check add constraint FXVABGTWKSVZXOBC foreign
key(systemeventid) references systemevents(systemeventid) on delete cascade;
93. alter table autodiagnostic with check add constraint FXYGHGSHNSBGKZKP foreign
key(systemeventid) references systemevents(systemeventid);
94. alter table activehosterrors with check add constraint FZOGNHGXVTVCZBKB foreign
key(startsystemeventid) references systemevents(systemeventid);
95.
96. *** Command Dumping >>> Recreate uniquekeys for other non-primary key indexes
97.
98.
99. *** Command Dumping >>> Recreate any leftover nonkey indexes
100.
The result for the command dumping “*** Command Dumping >>> Drop foreign keys” must have a total of 11 drop
constraints for foreign keys list as shown in above output.
The result for the command dumping “*** Command Dumping >>> Drop other indexes” must have a total of 9
dropped indexes list as shown in above output.
The result for the command dumping “*** Command Dumping >>> Drop primary keys” must have a total of 17
dropped constraints for primary keys list as shown in above output.
The names of keys and indexes could be different.
If your output is exactly what is shown above, it means that your database integrity check is passed. You may now safely
proceed with the Pre-requisites before migration section of this article.
If your database shows less than the mentioned output, the database integrity is in unstable condition. Contact Asiainfo
Technical Support for further troubleshooting. Do not proceed with upgrading the Deep Security Manager.
In order to verify if the relationship between tables is correct, check and match the database diagrams:
4. Choose the following table pairs for checking their relationships and then click Add.
There are 7 sets of tables as mentioned below and their respective relationship diagram is also shown. Please
make sure your database relationship diagram also matches the same as shown below for each set.
SET 1
systemevents
activehosterrors
agentevents
autodiagnostic
systemeventhistory
SET 2
antimalwareevents
antimalwarequarantinedfile
antimalwarespywareitems
antimalwareeventhistory
SET 3
webreputationevents
webreputationeventhistory
SET 4
integrityevents
integrityeventhistory
SET 5
loginspectionevents
loginspectioneventhistory
SET 6
packetlogs
packetlogdatas
packetloghistory
SET 7
payloadlogs
payloadlogdatas
payloadloghistory
Pre-requisites before migration
2. Check the table usage and free disk space of SQL server.
b. Reserve another 150% space for migration purpose. For the example above, you need at least 150 GB free disk space to
perform the migration script.
3. Change the SQL server remote query connection timeout to "0 (No Timeout)".
4. You may opt to truncate data from the migrating tables to improve the migration speed and reduce the disk space usage.
The action is TRUNCATE TABLE, which will clear the table without any records.
a. Modify the following code to specify which part you want to truncate. By default, it will truncate all the migrating tables.
You can comment the default one and remove the SQL comment on the part you want to truncate.
For instance, if you would like to truncate integrityevents related data, modify the code similar to the following:
Truncating the systemevents table will also remove the following tables for data consistency:
Activehosterrors
Agentevents
Autodiagnostic
Systemeventhistory
2. Open the script and modify the database name for each tenants. For example, if your database name is DSM95, change the
script to [DSM95] similar to the following:
use [DSM95]
go
3. Stop the Deep Security Manager service. If you are in a multi-node environment, stop the DSM service on all DSM nodes.
4. Run the script using the SQL Management Studio console or the SQLCMD tool.
b. Copy the modified SQL script into this new query and press CTRL + A to select all.
e. Copy the modified SQL script in C:\ drive of the DSM database server.
f. Open a CMD command window and switch the current path to C:\.
If username and password are required, use –U $username –P $password to appoint the username and
password.
6. If you have a multi-tenant environment, run Steps 2 to 4 for each tenant database.
7. Once the migration script is completed, run the DSM installer to upgrade.
Troubleshooting procedure
If the script failed for any reasons (e.g. space or connection timeout), do the following:
1. Troubleshoot the issue using the SQL error or any MS SQL server log.
2. Once the issue has been fixed, restore the database and run the script again.
To upgrade older Deep Security Manager builds to 9.5 SP1 Patch 2 build 9.5.3.6511, the process will pass you through two
mandatory upgrade steps:
1. Upgrade the database schema first, using those scripts mentioned in above section.
c. Database Schema Upgrade message will appear asking for appropriate selection as shown:
d. From above screen, if you have already performed the Database Schema Upgrade through script, and have
checked the box under “Please Confirm” section, the installer will continue as usual and will upgrade the DSM.
e. If you have not performed the schema upgrade process mentioned previously in this KB, have checked the box
under “Please Confirm” section, and click NEXT, the following screen will appear.
f. The installer will not continue and will exit when you click OK.
Notes
It is not necessary to apply this update procedure if you are using Oracle Database.
In a multi-node environment, you need to run the upgrade installer separately. However, if Node 1 has been upgraded and
schema change has already been done, upgrading the Node 2 will be the usual upgrade process.
In a multi-tenant environment, you have to upgrade the database schema for each tenant and then run the DSM upgrade
installer.
In some cases, the DSM installer may not ask for upgrading the database schema before upgrading the DSM. This is because
the already installed Manager’s database has already been upgraded for required schema changes. As an example, when
upgrading the DSM from 9.0 SP1 Patch 5 to 9.5 SP1 Patch 2 and the installer does not ask for schema upgrade, you may
proceed upgrading the DSM safely.
It is not recommended to do the DSM upgrade in silent mode. If you are using silent mode, the database schema upgrade
window will not appear.