System migration. Stages of the Data Migration Process in IC Implementation Projects

The service for the migration of the existing IT service management automation system from the HP OpenView Service Desk 4.5 / 5.x product is aimed at solving the following tasks:

• reducing operational risks by switching to a product that is fully supported by the developer and his partners in the Russian Federation;
• reducing migration risks, reducing the cost and timing of migration by switching to a solution that is ideologically as close as possible to HP OpenView Service Desk 4.5 / 5.x (does not require major revision of existing processes and complete retraining of personnel);
• ensuring the possibility of further development of IT management practices by overcoming the functional limitations inherent in the current automation system.

The migration is carried out to a specialized design solution OMNITRACKER CleverENGINE, implemented on the basis of the OMNITRACKER platform. This solution provides customers with the following important benefits:

• CleverENGINE was designed from the ground up to match the HP OpenView Service Desk as closely as possible to ensure smooth migration. CleverENGINE has a similar data model and the same user interface organization principle. This allows during migration to save manpower and funds for the forced adjustment of existing management processes, retraining of personnel, and also leaves the possibility of transferring historical data;
• CleverENGINE solution was designed and developed by existing IT management consultants. The functionality built into CleverENGINE is vastly superior to the HP OpenView Service Desk. In particular, it provides visualization of the CMDB, accounting and control of the use of software licenses, release management, maintaining a knowledge base, automation of routine operations, a powerful mechanism for approvals, and much more. This allows customers to improve their IT management practices, overcoming the functional limitations of the existing automation system;
• The OMNITRACKER platform is an environment for the rapid development of various workflow applications (also outside of IT service management). As a rule, solutions based on OMNITRACKER are handed over to customers in an open form - available for adaptation and further development. Many customers develop their automation systems on their own with little or no help from partners. At the same time, on the basis of one platform, you can gradually develop the functionality traditionally inherent in various products - service management, project management, document management and other solutions.

As part of the migration, a survey of the existing automation system is carried out, and CleverENGINE introductory seminars are held for the customer's key personnel. Based on the results, the requirements for the final solution are clarified, opportunities for gaining quick benefits are identified through the use of ready-made CleverENGINE functionality that was not previously available. Then the construction of a new automation system is carried out, qualification tests are carried out, and personnel training is carried out. Further, if necessary, the transfer of historical data is carried out and a switch to the new automation system is carried out. After that, the customer is provided with technical support during the trial operation. At the end of the project, the automation system is transferred to industrial operation.

results

As a result of the implementation of the migration project, the customer receives:

• a new automation system while maintaining the control processes that operated in the old automation system;
• new functionality of the automation system, previously unavailable;
• a set of Russian-language user documentation for a new automation system.

Typical project profile

The duration of the migration project is from 2.5 to 4.5 months - depending on the degree of “customization” of the existing system based on HP OpenView Service Desk (the number of business logic rules, forms and data views, reporting, integration solutions, etc. ), as well as from the need to transfer historical data. The project is carried out by 1-2 specialists. The work involves specialists with expert knowledge of both products - both HP OpenView Service Desk and OMNITRACKER CleverENGINE.

The service for the migration of the existing IT service management automation system can be performed for systems implemented on the basis of various software products. Depending on the product from which the migration is performed, the customer will receive different benefits, the project will require different resources. At the same time, the general logic of the organization of work is preserved.

Examples of completed projects

Examples of projects completed by Cleverics consultants:

• Migration of the ITSM process automation system from HP OpenView Service Desk to OMNITRACKER CleverENGINE and optimization of user support activities (SK Alliance, 2014)
• Migration of the ITSM process automation system from HP OpenView Service Desk 4.5 to OMNITRACKER CleverENGINE (KATREN, 2013)
• Migration of the ITSM process automation system from HP OpenView Service Desk 4.5 to OMNITRACKER CleverENGINE (VTB24, 2012-2013)
• Migration of the ITSM process automation system (AutoSpecCentre, 2010)

Any business strives to spend as little as possible, including on IT. One of the ways to save money is to move information systems to the cloud. When transferring data to a virtualized environment, it is necessary to take into account the requirements of the legislation on the storage and processing of personal data. These requirements apply to any organization.

Vladimir Lebedev, director of business development at Stack Group, told us about the intricacies of the migration process.

Legal requirements

In 2006 was adopted 152-FZ "On personal data", which is designed to protect individuals in the automated processing of personal data. Last year, a package of amendments came into force on the localization of personal data in Russia, which, according to the authors, should increase the level of information security within the state and stimulate the Russian market for technological solutions and the information security market.

By law, a business is obliged to collect, store and process personal data on the territory of the Russian Federation. All requirements are exactly the same for both Russian and foreign companies if their activities are aimed at the territory of Russia. At the same time, it is possible to transfer personal data outside the country, but they must be immutable, and their volume must not exceed the volume in Russian databases.

For whom is the law?

Each organization acts as an operator of personal data, for example, its employees, customers, partners, therefore all companies in all industries fall under the law. However, potential inspections primarily threaten companies that have commercial activities based on the processing of personal data: banks, insurance companies, companies providing medical services, and the like.

The Roskomnadzor audit plan for 2016 includes: the largest software companies, international banks, chain trading companies and online stores.

Difficulties of transferring personal data for international companies

Often, global international players, in order to comply with legal requirements, are forced to transfer to Russia not only the data itself, but also all related systems: it is extremely difficult for companies with centralized IT systems to change the architecture of an information system under one local legislation, because initially there were no separate information processing centers. Rebuilding information systems requires substantial investment.

Virtualization

Moving to the cloud is less expensive than buying and installing hardware. At the end of 2014, prices for Russian clouds were on average 15-30% higher than for European ones, and at the end of 2015, on the contrary, our prices became 20-30% lower: the exchange rate and the relative cost of placement on Russian dates changed. -centers.

Operators began to offer a more mature product, and foreign companies were able to choose among virtualization systems, price segments, and opportunities appeared for the connectivity of Russian and Western infrastructures.

For foreign companies, virtual infrastructure is a reliable and understandable solution. Many companies are transferring information exclusively to a virtualized environment, in which the necessary means of protecting personal data can also be placed. When relocating, foreign companies, of course, expect the operator to comply with the information security requirements of the regulators.

Storage Migration Risks

The migration processes themselves from one system to another carry risks. First of all, the established level of customer service may be violated. Communication may also be interrupted and the work of units whose activities are tied to automated systems may be interrupted, conditions may arise when information security requirements cannot be met, and other technical problems. Therefore, it is important that providers are prepared to ensure business continuity with reliable and proven IT migration solutions.

Cloud migration stages

We will consider the general principles of service migration, that is, transferring the operating systems responsible for the operation of this service to a virtualized environment, using the example of a solution VMware vSphere.

It is desirable to divide the architecture of the company's information systems into services that perform specific tasks. Usually, everything is moved to the cloud, except for services that are incompatible with the cloud technology for technical reasons (RISC architecture) and those that cannot be moved due to a license.

Further it is necessary information systems audit... At this stage, the composition of services is determined (which operating systems are related to a particular service), as well as their connectivity. The main difficulty lies in the variety of source operating systems and the physical architecture of the servers on which they run. Based on this information, a migration plan is drawn up, taking into account the current business processes: the requirements for the connectivity of physical and virtual infrastructures are determined, the order of migration, and the permissible "migration windows" are set. It is important to remember that you cannot update versions of software products or operating systems during migration. Simultaneously with the migration, only the revision of computing resources (CPU, RAM, HDD) is allowed.

As a rule, the utility is used for migration VMware converter, which works effectively when transferring an OS of the Microsoft Windows family (but the migration of services running in these OSs has its own nuances). But due to the peculiarities of Linux file systems, in about 40% of cases, after the VMware converter finishes working, the virtual machine may not start. If LVM is used on Linux, then you need to deploy a new OS instance from the provider template in a virtual environment and then migrate data, software products, and internal services.

For any type of OS, there are general conditions that complicate migration: firstly, the way of storing data, because of which direct migration is impossible, is dynamic disks in Windows or LVM in Linux, and secondly, difficulties due to the use of software and hardware RAID arrays. Thus, even accurate data transfer alone does not guarantee that the virtual machine will start successfully. On a physical server, the operation of virtual machines is provided by a hypervisor - an OS that divides a physical server into several virtual machines that can work simultaneously and use the same physical resources. Naturally, the set of virtual hardware at the hypervisor does not match the hardware of the physical server on which the OS was running before migration. Accordingly, due to the difference in drivers, there are many differences in access to this equipment.

Migration of ADDS and MS SQL without stopping services

Business almost always requires a number of services to remain available during migration. Moreover, often migration without stopping the service recommended as the most reliable. Therefore, we will consider the features of migration without stopping the most popular services of Microsoft OS: Active Directory Domain Services (ADDS or AD) and Microsoft SQL (MS SQL). The following algorithm is used to migrate Active Directory without stopping the service:

• Network connectivity is formed between the physical hardware and the virtualized environment. Typically, this is a site-to-site VPN - it creates a logical network on top of another network. In this case, traffic can be protected by encryption using IPsec protocols.
• In the cloud, we deploy new virtual machines from a template, where we configure AD domain controllers and add them to the forest.
• The Active Directory database is replicated over the network via VPN from working controllers on the side of physical equipment to the cloud.
• After data replication, we reassign the operations role masters to the cloud controllers and remove the domain controller roles from the servers.
• Then we check the operation of the services and disable the accounts of the old controllers and physical equipment.

Algorithm MS SQL migrations more complex, since MS SQL is usually used in a multi-tier service as a backend. In the DNS records in applications that use databases (MS SQL clients), you have to manually specify the new location of the database. Therefore, the simple cannot be completely ruled out, but it can be minimized. There are mechanisms for non-stop MS SQL migration, these include Mirroring and AlwaysOn, but their use is not always justified. AlwaysOn is only available in expensive Enterprise editions, and Mirroring must be supported by MS SQL clients. In addition, to use the Mirroring mechanisms, additional configuration is required for all MS SQL clients.
Let's consider the most common option for migrating MS SQL to the cloud:

• Configures network connectivity between the cloud and physical equipment.
• We make sure that the MS SQL database recovery model is complete, then you can make and transfer a full backup, and then synchronize both databases, transferring copies of the transactional logs.
• In the cloud, we deploy a virtual machine from a template, in which we install and configure a new MS SQL server.
• We create a full backup of the MS SQL server database running on a physical server, then restore it to the cloud, while the method of transferring the backup depends on the file size and network bandwidth - we move it on a physical medium or copy it over the network.
• After restoring the database in the cloud, we make a copy of the transactional logs and also restore them in the cloud.
• During the "migration window", we stop the MS SQL server running on physical hardware, create and restore the last minimal copy of transactional logs in the cloud, launch the MS SQL server in the cloud and switch clients to the new location of the database.
• We check the work of services, turn off the physical equipment.

For each service and service, there are many ways to migrate, depending on the conditions. A service provider can help you avoid the hassles that may arise during migration.

Information Security

Companies must not only transfer data to Russia, but also ensure an appropriate level of information security for their storage.

In the list of documents defining the requirements and describing the measures necessary to protect information systems, in the first place is, of course, the law itself, which establishes the general requirements. Specific actions are regulated by decrees of the Government of the Russian Federation and regulatory documents of the FSTEC (Federal Service for Technical and Export Control) and the FSB (Federal Security Service).

Requirements for the storage of personal data

Requirements for the technical protection of confidential information and the provision of information protection services are clearly formulated. The tools for their implementation are diverse. In particular, it can be firewalls, intrusion detection systems, security analysis tools, anti-virus protection, protection tools for virtualization environments. The market offers a wide range of information security tools - both Russian and foreign vendors. There is already law enforcement practice, since the law has been in effect since 2007. In general, the approach to regulation in Russia differs from, for example, the European approach. For example, in Russia, failure to comply with the prescribed information security requirements leads to liability. And in the West, a company can independently determine in what way to fulfill the requirements, and responsibility comes only if illegal actions are committed with personal data.

Infrastructure Requirements

Regulators also impose certain requirements on infrastructure. For example, there is certification of built automated systems that ensure information protection... The certificate is issued by an independent laboratory, which confirms that the infrastructure is ready and meets all the requirements for the security of information systems. This certification is not required for commercial organizations, but it makes it possible to understand that the data center has the necessary protection and is able to ensure the security of the company's data.

There are also international ISO standards governing the construction of an information security management system ( set of ISO 2700x standards). Many foreign companies meet these standards.

In addition, a data protection outsourcing model is applied. Companies providing such services must have the necessary licenses and competencies.

INFO

Transfer forecast

It is quite difficult to calculate exactly how much data is to be transferred to Russia, but based on the occupancy of the data center market, we can say that there is enough capacity to localize data in accordance with the law. For example, in the market of the Moscow region, there is an oversupply of capacity: the total capacity is about 27 thousand racks and almost 40% of them are free. Many data centers have high availability areas. It should also be taken into account that the data density in a single rack can vary depending on the equipment. Today, one unit in the server rack processes significantly more information than it did several years ago.

At the same time, the demand is uneven: a year ago there was a rather serious rise, now the number of requests from foreign companies has decreased. Some companies have already moved and meet the requirements, and some are waiting, for example, for the results of inspections by regulators (they will influence the formation of the second wave).

A lot has changed in the information technology world in the past 32 years since Microsoft released Windows 1.0. The only thing that remains the same is the complexity of the migration processes, or the transition to a new version of the operating system, and the deployment of updates. When you ask users what they want from the migration, you get the answer: a smooth transition with minimal downtime and a familiar desktop experience. Some will argue that they are generally against migration, but there are usually few of them.

What is migration for

There are many reasons for migrating at the desktop level. The two main ones are safety and operating cost. Microsoft developers have steadily improved the security of desktop operating systems; the list of security technologies that were not present in older versions is impressive. Threats are becoming more sophisticated, and Microsoft had to add features such as launching an ELAM antimalware solution at the first download stage (https://docs.microsoft.com/en-us/windows-hardware/drivers/install/early-launch-antimalware ) and controlled folder access (https://docs.microsoft.com/en-us/windows/threat-protection/windows-defender-exploit-guard/controlled-folders-exploit-guard), an excellent way to protect against programs- blackmailers, in itself a sufficient reason to think about migration. Security is a particularly serious concern for companies still running versions of the Windows 7 operating system (and even Windows XP), as threats have changed significantly since then, and the improvements made to Windows 10 represent in some respects the most basic and not always sufficient. security level.

Operating costs are another important factor; Migration is often tied to hardware and Microsoft Office toolkit update cycles, so more and more companies are considering virtualization to break their dependency on hardware and desktop operating system updates. Even companies looking to retain physical hardware can attract potential savings from running the current generation of Windows on modern hardware.

Migration process

The figure below shows a simple diagram of the main steps in the migration process.

There are many different approaches to how to carry out each step; whole books have been written on how to carry out, for example, pilot IT projects. Let's take a quick look at each stage from a migration perspective.

Environment inventory

Many companies have advanced inventory and management systems that can provide answers to questions such as "How many heavily used desktops are running Windows 8.1 SP2?" or "How many Dell XPS 13 laptops are in stock?" Management systems such as System Center Configuration Manager (SCCM) and Microsoft Intune can be leveraged for inventory if you have the resources and have deployed them. Otherwise, you might have to resort to less efficient methods, such as soliciting user feedback, preparing an inventory tool and promoting it through Group Policy, or manually inventorying and configuring system settings. The key condition at this stage is to obtain an accurate model of existing systems in order to properly plan the desired state.

Desired state design

The term “designing the desired state” implies the search for the kind to which we want to bring our environment in the future. Moreover, at this stage, a long-term plan is being built to implement your intentions. The plan for achieving the desired state of the migration should include the following information.

• What are you migrating from your origin (environment inventory required).
• What do you carry to your destination. There are many questions to be answered, including: Which version of Windows are you migrating to? Are you staying on physical desktops, moving to an on-premises virtual environment, or moving to the cloud? How will the target environment be configured? What to do if migration fails or systems cannot be migrated?
• Your schedule. Most companies have to migrate over an extended period rather than in one giant leap. When should you translate certain users? What does the overall timetable look like and how does it reflect external events (for example, the beginning of the school year, the end of the fiscal year, or the sales season)?
• Your business. Are you migrating to achieve specific goals? There may be line-of-business applications that need to be migrated, or there may be special regulatory or compliance requirements for the new environment.

The purpose of this step is to help you understand what the capabilities of the completed environment should be and how to move from your current situation to the desired position. It is impossible to spend too much time or pay too much attention to detail here. The old adage - measure seven times, cut once is the best guide to action. This step usually does not require any changes to the environment.

Getting Started with Prototyping

You now begin building the environment described in the desired state design phase. Imagine that you are building a house: you start on an empty piece of land, dig out the soil, pour concrete for the foundation, and start building the walls. There is a certain minimum amount of work that must be done in order to move into the house, but you can live in an unfinished house. The same is true for migration: before a pilot can be carried out, at least a portion of the new environment must be designed, built, and tested. For example, internal deployments of VDI virtual desktop infrastructure will need to create an appropriately sized hypervisor environment, obtain the correct licenses, and so on, before a single desktop can be migrated in a pilot project. But it is not necessary to completely build the entire desired state environment. This phase is often scheduled to match the deployment of the various components with the actual migration.

One of the advantages of the construction phase is the convenient opportunity to include additional precautions in the schedule. The point is that depending on the nature of the migration, a small number of users in a pilot project can usually be migrated to a new platform at the outset. This allows you to proactively monitor your environment and fix any issues that arise, and experiment with tools and settings before all pilot users get them.

In addition, during the construction phase of the prototype, it is convenient to eliminate the deficiencies of the existing environment, which should be eliminated before starting the deployment.

Pilot stage

During the design of a new aircraft or rocket, engineers plan an extensive flight test program. Each test flight has a specific goal, which can be simple (test landing gear extending and raising at different speeds) or complex (maneuvering 2v2 fighters). Test pilots do not improvise during flights, they strictly follow the assignment for each flight, achieving their goals. Your pilot should work the same way: specify what needs to be proven, tested, or verified, and then make sure that all activities in the pilot are focused on achieving that goal. Too often, a pilot program really means "pick a few random users and see what goes wrong." A good pilot program starts with a structured test of a set of basic functionality — migrating a single user, migrating a group of users, testing the resulting environment for day-to-day tasks, and then moving on to more complex scenarios. Structured testing is especially important when you are moving from a traditional desktop environment to the cloud or virtual elements. It will take time to learn how to manage the cloud or hypervisor, including getting vendor support in the event of a pilot problem.

While this is not considered a separate step, this is the right time to introduce users to the new environment. It would be great if you could deploy a new desktop environment without training users, but in reality, you need to take time to familiarize users with the specifics of the environment. With a stable pilot program, you can run demos, record instructional videos, or provide users with a sandbox with limited access to the new environment during the learning process. Keep in mind that support group technicians are likely to need training as well. It is possible that additional staff will be required to cope with the increased flow of service requests: “Can't find feature X” is a typical complaint in the early stages of migration.

Deployment

The main events take place during the full-scale deployment phase. This migration phase is not much different from any other complex technical project: you define a schedule for migrating different groups of users, and then follow the planned steps, solving problems as they arise. If the prototyping and pilot phases are done correctly, you shouldn't have major new problems. As a general rule, it is best to be careful when designing this stage; if you plan on moving 100 desktops a week, and in reality you manage to transfer 150, that's better than planning 150 and doing 100. Don't forget about holidays, vacations, and other personnel restrictions. For a successful deployment, keep in mind that you are migrating an environment that your users depend on, so you need to do this in a way that minimizes the inconvenience of your users.

By the time production begins, the support team should be staffed, trained, and ready to support the number of users migrated each week; your new environment should be stable and productive, and pilot users should be able to work seamlessly in the new environment. If any of these conditions are not met, this should serve as a warning: stop and solve the problems before starting a full-scale migration.

Exploitation

The exploitation phase is a bit boring as it forces you to go back to where you started: regular support and management operations in a stable environment. This phase is the day-to-day maintenance work for the new environment, including fixing user issues, testing, and applying regular vendor security updates. The pilot phase is like an access road to the migration process, and this phase is a motorway; you are running smoothly at a constant speed (at least until the next major update from Microsoft).

Simplifying the process

The right choice of tools will significantly simplify the pilot phase and release to production. Users should be able to work with minimal disruption, and the best way to meet this need is through thoughtful design, profile migration, and managed solutions from different companies. For example, Liquidware offers a Workspace Environment Management package to help you complete every step of your Windows migration, including assessment and design with Stratusphere, and migration with ProfileUnity User Management. Stratusphere detects installed and used applications and helps you evaluate and provision hardware for Windows 10. With ProfileUnity, you can seamlessly migrate user profiles between any version of Windows and significantly reduce the time required to migrate. The Profile Bridge technology used in ProfileUnity allows you to place user profiles in containers and work with them without problems on different versions of Windows, compatible both in the opposite direction and in the forward direction. ProfileUnity reduces move and sync times, and you can deploy to desktops, laptops, virtual systems, and the cloud to provide users with a familiar experience on all devices at the same time.

Most modern Companies use various information solutions in their activities, and for almost every Company, sooner or later, issues of information systems integration and / or migration to more technological platforms become relevant. There are often no ready-made answers to these questions, so IDelync has a comprehensive and balanced approach to choosing the best solution options for each Company.

Integration of various information solutions allows the Company to:

organize work with all levels of corporate data, regardless of the type of software products;

exclude or minimize duplicate input of information into the existing information solutions of the Company;

to organize a single information space of the Company, with clearly defined flows of information between different solutions, differentiation of rights between users.

The main advantage of integration solutions is that the Company can obtain a positive effect from their use in a shorter time frame and at lower costs than when choosing solutions that imply the replacement or modernization of information solutions. IDelync specialists have extensive experience in integration and have implemented projects of various information systems, such as: 1C 7.7 -> 1C 8.x; Info accountant -> 1C; Turbo Accountant -> 1C; Sail -> 1C; StoreHouse -> 1C; R-Keeper -> 1C; Excel -> 1C.

Migration of systems to more technological platforms or solutions is carried out when certain information solutions are outdated and no longer meet the company's requirements. At the same time, the Company needs to make the transition quickly and efficiently. It is these high-quality and time-optimal solutions that IDelync specializes in. In addition, our company has developed a unified integrated module for accounting systems based on 1C: Enterprise, namely from 1C: Enterprise 7.7 to 1C: Enterprise 8. As a result of migration, the Company receives not only a more modern, productive solution, but can also automate processes, not previously covered.

IDelync proposes to migrate to other software products in stages, going through the following stages:

Architectural, analytical and business audit of existing IT systems. These works are performed to determine the current state of systems and obtain information for choosing a new system. During the same period, the collection of business requirements for the new solution is carried out, which makes it possible to provide the necessary modules in the new solution.

The choice of technologies for the migration of IT systems. At this stage, the selection and design of technologies for performing migration work is carried out. If you plan to migrate from various configurations of 1C: Enterprise 7.7 to 1C: Enterprise 8, IDelync uses its own module, tested on many projects.

Development and configuration of the migration mechanism and its testing.

Migration to a new solution, trial operation of the resulting IT-system.

Putting IT systems into operation, system support. Careful planning of work related to the commissioning of a new system allows the launch of a new solution on time and minimizes the risks associated with the implementation of a new solution.

System support is carried out under a separate agreement with the Customer or the Customer's own IT service.

Thanks to a thorough collection and analysis of the Customer's business requirements, the design and testing of systems integration and migration mechanisms, and clear interaction with the Customer at all stages of work execution, IDelync successfully integrates and migrates systems both within individual projects and as part of integrated operational automation projects. management accounting of companies.

More services (3)

More solutions (1)

More experience (3)

• "ID.Integration": integrated data exchange module

  Many enterprises have now achieved a high degree of automation. In most cases, the Companies do not possess any single information solution, but are provided with a set of various systems that are used for local accounting tasks for maintaining accounting and tax accounting, operational and management accounting, as well as obtaining the necessary reporting.

  A typical product, on the basis of which information solutions of domestic enterprises are made, is 1C. In this regard, the following questions can be singled out that most frequently arise among the enterprises of the Company:

  Integration- the need to combine several separate systems to obtain a single information solution and eliminate the need for multiple data entry into separate information databases;

  Migration- implementation of a quick and high-quality transition from solutions of any versions of the accounting system to systems of version 1C: Enterprise 8.
  

  To address these issues, IDelync specialists have developed a universal, integrated module for data exchange between various accounting programs and configurations of "1C: Enterprise" platforms version 7.7 and 8.1.

  The integrated module provides ample opportunities for flexible customization:

  Synchronization of reference data objects, including the ability to combine several objects from the source into one receiver object to obtain aggregate data, as well as to compare objects from different source databases to a single value in the receiver.

  Synchronization of PS, postings and correspondence - to reflect business operations.

  Using a universal data exchange module allows you to correctly solve the following tasks:
  

  Merging data from multiple accounting databases into a single consolidated system.

  Transformation of accounting transactions both into accounting transactions of other types (Tax, International, etc.), and into accounting transactions with other correspondence, depending on the analytic objects.

  Full automation of the data exchange process. In accordance with the preliminary setting of the tables of comparison of reference data, PS and correspondence, an automated process of data exchange is carried out, after which it is quite simple to check the correctness of the loaded data.

  The ability to carry out manual data exchange with flexible pre-setting up to a separate operation of reflecting the accounting of economic activities. At the same time, on the side of the receiving base, it is possible to manually correct the postings of loaded operations with the condition that changes are not corrected when the data on this operation is updated again.

  It should be noted that the use of the integration module during the transition of an enterprise from systems based on 1C 7.7 to 1C 8.1 allows not only qualitatively and quickly transferring the "cut" of balances (ie balances on accounts on a certain date), but also providing a complete transfer turnovers on accounts for any period of data available in the current accounting system. This approach allows for the transition from one accounting system to another while maintaining the possibility of generating regulated reporting for the entire reporting period, regardless of the date of the start of operation of the new system.

Wholesale and retail trade, E-Commerce

• Automation of management accounting and optimization of the business process of implementation and logistics for OOO "Trading House" Slobozhanka "

  The company LLC "Trading House" Slobozhanka "(hereinafter" TDS "," Enterprise ") is an importer and distributor of famous brands of cosmetics, hygiene products and household goods. Using a well-developed structure, TDS sells in bulk to its customers a wide selection of cosmetic and hygiene products of well-known brands from manufacturers in Poland, Russia, Ukraine, and China.

  The objectives of this project were to increase the efficiency of the operational and management processes of the Enterprise in terms of the accounting function, management and control functions through automation.

  Taking into account the continuous operation of the Enterprise, as well as the rather complex logistics issues associated with the need for a quick high-quality organization of transport loading, the main task to be solved in terms of operational accounting was the optimization of order processing, preparation of documents necessary for shipment, loading of cars and their dispatch.

  Based on the results of the preliminary survey, based on the analysis of the business requirements of the Enterprise, the Consultant recommended a transition from the Granit information system to an ERP class system based on 1C Enterprise 8. The 1C configuration "Manufacturing Enterprise Management" was chosen as the basic solution. In a unified system of operational, managerial accounting, a business process for managing customer orders has been implemented, which includes the tasks of preparing documents, transport and warehouse logistics and controlling the confirmation of shipment. To optimize the process of accepting customer orders, the system implements a wide range of integration options with external sources (Emigo, Exite, EXCEL, PDA). The system also implements the process of pricing by brands and customers, as well as a mechanism for monitoring accounts receivable by brands and terms. The implemented solution covers all blocks of the enterprise's economic activity and also allows you to keep records of fixed assets, calculate management salaries, manage customer relationships (CRM) and receive financial statements for any period of interest (including on a daily basis). The following reports are implemented in the financial reporting block: Statement of Financial Results, Statement of Cash Flows, Balance Sheet. The unified management accounting system is integrated with accounting systems for the subsequent preparation of regulated reporting, integration with the Client-Bank system has been carried out.

• Technosila, Innovation and Reengineering for E-Commerce, Technosila merger with Technoshock group

  Technosila, Moscow, Russia, http://www.tehnosila.ru, one of the leaders in the retail chain in the field of electronics and household appliances in Russia. 50 regions, 137 stores and an online store covering all regions.

  Projects: IT Audit of information systems and business environment (SAP, 1C, PHP, FoxPro, Delphi); Management of IT Projects and business innovations for e-commerce areas; Reengineering of business processes for e-commerce; IT architecture design for e-commerce, integration and online interaction; Release of IT Projects for e-commerce; Merging of Technosila with the Technoshock group, http://tshok.ru, St. Petersburg, Russia, 16 cities, 42 stores and an online store.

• • Integration of MS CRM (Microsoft Dynamics 4.0) and 1C.8.1 "Manufacturing Enterprise Management" for a group of Internet companies SUP and + SOL, Moscow, Russia

    The purpose of this Project was the development and implementation of an application that solves the problem of information exchange in the established format between the specialized information system Microsoft Dynamics CRM 4.0 and UIS based on 1C: 8.1 "Manufacturing Enterprise Management".

    The implemented mechanism carries out a two-way exchange of information between infobases on-line. As a result of each data exchange session, the UIS receives updated normative and reference information about customers, nomenclature, client applications, and in the opposite direction (to MS CRM) information about payments issued on the client's documents is transmitted.

    The design and implementation of the data exchange mechanism were implemented by the Consultant in close cooperation with a third-party implementer of the customer relationship management (CRM) information system based on the Microsoft Dynamics CRM 4.0 product. A clear organization of work on interaction with a team of a third-party developer made it possible to design, implement and implement a data exchange mechanism in a short time and provide the Company with the necessary, consistent information in both systems.

In this article, we would like to systematize our experience of data migration in large corporate projects related to the transition of Customers to work in 1C: Enterprise 8 configurations.

In this case, the main emphasis in the article will be made, first of all, on the technological component of the migration process. The organizational component is also affected, but to a lesser extent.

Terms and Definitions

By data migration, it is customary to understand the final sequence of work, a project aimed at a one-time massive transfer of data from source systems (historical systems) to a receiving system. At the same time, the exploitation of this data in the source systems is terminated.

Distinguish data migration from data integration. Integration, as opposed to migration, is a permanent part of the IT architecture, and is responsible for the flow of data between different systems and data stores - and is a process, not a project activity.

In general, the migration scheme looks like this:

Rice. 1

Historical systems- databases of the Customer's company, which are planned to be completely or partially replaced when introducing a new system.

Receiver system- target system, arbitrary configuration "1C: Enterprise 8".

Initial data- data unloaded from historical systems into an arbitrary xls -file format. In this case, the xls format seems to be one of the most convenient, since the ability to upload to an xls file is present in many accounting systems of "previous generations".

It is possible to consider the xml file format as a modern alternative as a transport.

There are also options for using the staging database.

Transformation, conversion- the process of converting raw data into data for loading. Data transformation occurs in accordance with the templates for loading. The transformation results in data for loading.

Download data- data intended for loading into the receiving system. In this article, as well as the original data, the xls format is considered.

Data templates to download- a description of the data tables to be loaded into the target system.

Migration stages

Let's consider step by step the process of preparing and carrying out migration.

The organizational stages of migration include the following points:

· Define a migration strategy. At this stage, the Contractor and the Customer agree on the technology for carrying out migration work;

· Determination of the composition of the working group on migration. The working group should include specialists of both the Contractor and the Customer, who are sufficiently familiar with the operation of historical systems (from the Customer's side) and the target system (from the Contractor's side);

· Preliminary migration plan. The migration plan will be adjusted several times during the project;

· Periods of dates of unloading data from historical systems, data volumes. Data slice periods for migrations, dates of test and final migrations. This information can be attributed to the migration plan;

· Composition of data to be migrated. Reference data, classifiers, transactional data, balances, turnovers, etc .;

· Issues of checking the quality, correctness and integrity of data during the migration process and at the end;

· Rollback issues in case of failures.

Let us dwell in more detail on the technological stages of migration.

Rice. 2

1.Preparing data loading templates

A data load template contains technical descriptions of the data tables for loading, algorithms and loading rules for the current template.

Each template is generally intended for one or more related tables in the target target system.

The template indicates:

Description of all fields of the xls data file to be loaded, including:

o Field name

o Sign of the obligatory filling of the field

o Example of filling out the field

o Note

Description of the rules for loading the table of the target system based on the data for loading (order in the case of several related tables, search algorithms for key fields, etc.)

· Description of filling directly the fields of the tables of the target system in case something other than transferring data "one to one" from the data file for loading is provided. Relevant for reference fields, for example.

In the course of work on this stage, the Contractor must also prepare a loader for data files for loading. In the case of working with xls files, this task is not particularly difficult.

2. Identifying data sources

This stage can start together with the previous stage “1. Preparing Data Loading Templates ”.

Within the framework of this stage, the Customer's specialists determine from which systems and which data can be downloaded. You should also determine what data Maybe may be needed.

As a rule, in large migration projects, identifying a complete exhaustive list of data sources can take quite a long time and occurs as you work in subsequent stages.

It is not uncommon for situations when, in order to ensure the integrity of information in the future, some data have to be transferred from printed sources (digitized) or even entered into tables from the words of the Customer's key employees.

However, at this stage, you should try to identify as much of the required data as possible.

3.Downloading raw data

The process of unloading data from historical systems can take a sufficient amount of time, especially if there are many systems, they are different and different departments of the Customer are responsible for them. It is necessary to take this moment into account during test and final migrations.

The most convenient option seems to be uploading to xls files. Many older IT systems support this option.

There may also be options for uploading to csv format, dbf, xml formats and others.

It is worth noting that for one reason or another (security issues, for example), the Customer cannot always provide data uploads in full at this stage! Only a data structure and a few test items. Thus, a situation may arise that during test and final loads, low-quality data will be found in the source tables, which will lead to unplanned errors.

To minimize this problem, the volumes of test downloads from historical systems should be discussed in advance.

4 data mapping

Mapping (data mapping) - in general, the process of mapping data from historical systems and the receiving system. That is, raw data and data to be loaded.

The mapping stage is the most time consuming stage and can take more than 50% of all work on the migration task.

At this stage, the entire working group of the migration project is fully involved.

In the process of data mapping, it is necessary to highlight the substeps of table mapping and field mapping.

· Mapping tables, or template mapping - mapping of tables of source data and data templates for loading. The correspondence can be either 1: 1 or N: N. As a result of this work, a register of table mapping is compiled and maintained. This sub-step is required for the next sub-step of field mapping and to keep track of the overall state of the mapping.

· Field mapping - mapping of table fields within the already defined mapping of tables. The result of this work is a register of field mapping.

As part of this stage, possible work on data normalization should also be carried out.

5.Prepare transformation rules

Unlike the previous stages, this stage is technical and involves the work of the Contractor's developer.

Based on the agreed registers of mapping fields, the Contractor's specialists develop data transformation rules.

For operational work during the preparatory migration stages and further, during test and final migrations, it is important that there is a convenient development environment for rules (scripts) for data transformation and an environment for converting source data into data for loading.

Moreover, the requirements for this environment include:

· Convenience and speed of development of transformation rules;

· Speed ​​of data conversion. Files in and out can be hundreds of thousands of lines!

· Ability to work with multiple input files at the same time;

· Ability to save transformation rules into separate files.

For our migration projects, we have developed a specialized developer workstation, taking as a basis the standard processing of the 1C Query Console.

Processing "Query Console" has been improved to make it possible to make direct requests to xls files.

Here is an example of combining two source xls files Employees.xls

and Operations.xls with pages:

Write-offs

and Receipts:

Note that the example is artificial, specially selected to demonstrate all possible stages of data source transformation.

The technological sequence of transformation operations here is as follows:

Using the Access SQL query language (which provides significant additional features compared to the 1C query language), an initial query is created that extracts data from the xls file into the 1C environment. At the same time, already at this stage, various checks and data normalization are possible.

ADO data access technology provides high speed performance.

Rice. 3

2. Query in 1C language - the main query that implements the field mapping algorithm. And also: enrichment of the loaded data with data from the 1C database, regrouping, combining with the results of queries to other source xls files, etc.

3.Post-processing of the 1C query result, if necessary. It is implemented using a script in the 1C language.

For example, here we implement the addition of the "TOTAL" line by the columns of the amounts.

4. Writing the final dataset to the xls file.

In the general case, at the output, we get the final files for loading into the target 1C database.

Also, this tool allows you to save the rules for converting data into a separate xml file:

In addition, the ability to work v batch mode, which is especially important when there is a large amount of heterogeneous migrating data.

In the course of the previous stages, the preparatory part of the work as a whole ends - all data sources have been identified, the source data has been unloaded from the sources, templates for loading into the target database have been prepared, data mapping has been prepared, and, finally, data transformation scripts have been developed.

It should be noted that before the final migration, you should definitely conduct several test ones. During test migrations, the Contractor together with the Customers identify:

Conversion errors, data loading errors

Conduct a preliminary assessment of the quality of data loaded into the target system

Based on the results of test migrations, draw up / update a plan for the final migration

7 data reconciliation

The quality check of the loaded data should be performed both after test migrations and at the end of the final migration. During reconciliation, the following indicators can be checked:

· Coincidence of the total amounts of balances, documents;

· Quantitative matches, for example, the number of fixed assets;

· Correctness of filling in individual sample entities;

Please note that certain checks of migrating data, data normalization issues must be addressed throughout all migration processes. You should always ask yourself what needs to be done at the current stage in order to avoid mistakes in subsequent stages.

For example:

· Checking for duplicates by key fields. It is possible and necessary to carry out more on the initial data;

· Casting of field types;

· Referential integrity;

· Mathematical inconsistencies. For example, checking for empty numeric fields, which are planned to be divided during transformation;

· In general, checking the mandatory completion of fields;

· Replacement of invalid characters. For example, English characters in Cyrillic fields ("o", "a", "e", etc.) This is especially true for key fields!

Checking the values ​​of string fields for compliance with the types of the receiving system (Length restrictions)

After the completion of the final migration, according to a predetermined migration strategy and migration plan, a decision is made on the further operation of historical systems.

Often, the operation ends immediately after the final reconciliation of data and recording the success of the migration - users of the new system no longer keep records in parallel in two systems, but completely switch to the new system. At the same time, access to the old system remains in read mode.

In some cases, parallel operation of two systems may occur during the trial operation (OE) and even more than this period. The issue of parallel users' work in two systems is closely related to the issue of the possibility of rolling back to the old system if the migration (or, in general, the work of the new system!) Is deemed unsatisfactory.

Conclusion

In conclusion, I would like to note that when it comes to migrating large transactional systems, which include many 1C: Enterprise configurations, the transition to a new system can be very time consuming.

Therefore, it should be remembered that any such project requires careful preparation and must be accompanied by an individual plan. However, regardless of the type of systems being migrated, the size of databases, etc., the overall migration scheme looks almost identical.