User Import & Availability  in ProfileMap

 

Version	Beschreibung	Autor	Datum
0.1	Initial Requirements	Guillermo Peiro	02.11.2022
0.1	Adding User Master data item “Country”	Georg Jülke	28.02.2023
0.1	Added current functionality explanation	Guillermo Peiro	03.04.2023
0.1	Added availability import	Guillermo Peiro	05.04.2023

Content

1 Content 3

2 User Stories – Overview 4

3 Overall Solution Design 5

3.1 Introduction 5

3.2 Business definition 5

3.2.1 US1 User creation 5

3.2.2 US2 user update 5

3.2.3 US3 user deletion 5

4 Architectural Design 6

5 Software Design 7

5.1 Data model 7

5.1.1 Use case 1 User creation 9

5.1.2 Use case 2 User update 9

5.1.3 Use case 3 User deletion 10

5.2 Import table 11

5.3 Availability import 11

5.4 Current functionality (Deprecated) 12

5.4.1 Use case 1 User creation 13

5.4.2 Use case 2 User update 13

5.4.3 Use case 3 User deletion 14

User Stories – Overview

The following list contains:

Use Case	Require-ments	Technical Design	UI	Backend	AI	Test Cases	Doku-mentation
UC1	Users Creation	x	N/A	x	N/A	x	x
UC2	Users Update	x	N/A	x	N/A	x	x
UC3	Users Deletion	x	N/A	x	N/A	x	x
UC4	Availability import	x
UC5
UC6
UC7
UC8
UC9

Overall Solution Design

Introduction

msg. ProfileMap requires data pipeline that allows the import of user data in the database. The data is private and must be consistent and updated according to what is provided by the Identity management of the client. A very important point is the removal of the data. Once the IDM once the user leaves the company the user cannot be part of ProfileMap databases anymore. In this document, the techniques for creation, update and deletion of the users will be described.

Business definition

In ProfileMap, an architectural system needs to be provided to import the users’ master data. The Pipeline receives data from the IDM (Identity Manager) via a file. The architectural design will then read this file to extract the employee’s data and add it to the persistency layer.

Strict requirements:

1. A user, who’s data has not been previously imported to ProfileMap persistency cannot register and will receive an error.
2. The user to be imported must be part of an organization with a GDP signed with ProfileMap.
3. A user that leaves the company or a business unit that belongs to the application needs to be removed from ProfileMap.
4. Users must be updated to reflect current real data (name change, location change etc.)

US1 User creation

The user creation will happen when a new user is added to the data import file and does not yet exist in ProfileMap database. The user creation extracts the data from the file and does an interface with ProfileMap data model (will be explained in the software design section).

US2 user update

The users have some of its fields from the file updated, the information should be extracted and updated. The user is identified as existing, and therefore need to be updated, by its id. A change in email will not be reflected and a new user will be created. The user business unit is another important field. On the change of business unit, the user needs to lose its permissions and roles from the previous.

US3 user deletion

Represents the most critical case. A user must be removed from ProfileMap once:

1. The user leaves the company (a leaving date is set upon the file, users sometimes got a backdated exit date, because the manager forgot to tell).
2. users just change their employee status from “Angestellter” zu “Freier Mitarbeiter”, will not be in the file anymore.
3. Users change their department and should not be delivered anymore (because they do not use profile map in the department)

Architectural Design

In development concept:

In the future concept, the process of updating the file does not change. However, the role of the lambda changes and now it just represents an interact between the content of the file (master data) and the admin API that in fact imports updates or deletes the data. Only the User Service API (update image!!!) has access to the users’ master data.

Field name	Description	Used	Required	Example
id	The unique identifier of a user if none is provided a UUID will be autogenerated	Yes	No	138507
firstName	The employee’s first name	Yes	Yes	Guillermo
lastName	The employee’s last name	Yes	Yes	Peiro Aznar
email	Identifies the user uniquely too. A user with a different email is always another user.	Yes	Yes	guillermo.peiro.aznar@msg.group
businessUnit	Extracted from the field “costCenter” the first 4 digits, identifies the BSU the user belongs too	Yes	No (inferred from “costCenter”)	1005
company	Company inside the main company of the client, used to describe a sub company in msg	Yes	No	msg systems ag
job	The general current role of the user	Yes	No	Senior IT Consultant
managerEmail	The email of the Line Manager	Yes	No, when no value and Not Specified will be assigned	Recardo.Jackson@msg.group
msgId	The id identifier in msg (maps to id)	No	No (when given maps to id)	138507
city	The current working city of the employee	Yes	Yes	Passau
costCenter	Defines the business unit for ProfileMap, can be useful for Informational porpoises.	Yes	Yes	100599020
importedAt	When the user was last created/updated from the pipeline	Yes	No	023-01-31T10:32:49+01:00
costCenterShortText	Description of the cost center text	No	No	QA Advisory & Analyt
personnelNumber	Internal number that defines the user in the client database	No	No	138507
namePrefix	???	No	No
initialEntry	The data when the user joined the company	No	No	44851
leavingDate	The date the user will leave the company	No	No	YYYY-MM-DD
companyCode	????	No	No
personnelArea	Internal business unit	No	No	org-gb-1312-195
personnelAreaText	Name/description of the business unit	Yes	yes	XQT – Test & Quality Management
personnelSubarea	Defines a sub area inside a business unit	No	No	org-b-1611-567
personnelSubareaText	Description sub area text	No	No	XQT – Crossbereich Management
employeeGroup	Type of employee(msg)	No	No	Angestellte
employeeSubgroup	????	No	No
objectAbbreviation	????	No	No
jobLevel	The level of the employee regarding experience	No	No	0
employmentPercentage	?????	No	No	Field name
country	The country where the user is located	No	No	ISO 3166-string | number | Alpha2Code | Alpha3Code

Software Design

The first thing to describe is the user data model that conforms the users master data. In ProfileMap we hold two types of data:

• Master data: Include everything related to the person that does not form part of his/hers CV profile. Among other organizational and personal information.
• Profile data: The profile data include information that forms the cv of the person in terms of competencies projects and experience.

In this document the description concentrates on the master data since no profile data is to be imported from the IDM sources.

Data model

In the table, we can find 5 different columns:

• Field Name: defines the name in the database the csv file must include it case sensitive.
• Description: short human description.
• Used: Defines if it is used by ProfileMap in any use case in the current data of the document
• Required: it is required or not to be included on the file.
• Example: real example for myself. 😊

Some fields are not used, and the description cannot be found as they come from Iportal original data model definition.

Use case 1 User creation

Creation process	Priority: Low	Estimate:
The user is created when a user with the same id or email is not found. Id and email are keys in the database and can’t be updated.
Acceptance Criteria The file is read. User that did not exist is added when newly appearing in the file.

The creation process does not include any special considerations. If the data owner (IDM) includes the data, it is allowing ProfileMap to add it the database. The user needs to be imported so it can be later registered. The user created will be the one whose id is not in the database.

Use case 2 User update

Update process	Priority: Low	Estimate:
Upon reading the file the user is updated with the current file information if its id or email already exist in the database.
Acceptance Criteria The file is read. User exists and information is correctly changed.

The special considerations to do when updating a user:

1. The user id cannot be updated because it is the primary key in the database.
2. If the user business unit/cost center is updated, the user loses all the management roles.
3. If the user was marked for deletion and no leaving data is specified, the mark “delete” will be set to false, because the user has been again imported in the file.

Use case 3 User deletion

Update process	Priority: Low	Estimate:
Upon reading the file the user information indicating the user has to be removed.
Acceptance Criteria The file is read. User is marked as delete for a period of 30 days.

This are the cases when the user must be deleted:

1. Users just change their employee status(employeeGroup) from “Angestellter” zu “Freier Mitarbeiter”, will not be in the file anymore and then marked for deletion.
2. Users change their department and should not be delivered anymore (because they do not use profile map in the department) therefor are not included in the file.

If some of the conditions above is true, the user will be marked for deletion and deleted after thirty days.

Field Name	Description	Used	Required	Example
toDelete	Boolean indicating whether the user is to be deleted	yes	It is inferred from the previous listed conditions	false
deleteAt	Date where the user should be deleted (current date plus 30 natural days)	Yes	Yes	03.12.2022

Two fields will be added to the data model:

Every time the pipeline runs it will check if a user is to be delete depending on the data deletedAt and if delete is set to true, if so, the API will be called to delete the user.

The risk of deleting a user incorrectly, is reduced because the period of 30 days gives flexibility of rollback.

Import table

The next image shows an example table in csv, the format of the csv must be semicolon separated and utf-8.

Availability import

In the case of the availability for the users, the import follows a similar process. A person is given access to a specific bucket with the IAM user “availabilityimport”, where they will upload a file and the import process will be triggered, the data model for the import is shown in the table below:

Field Name	Description	Used	Required	Example
month	The month we indicated the free pts for	yes	yes	2023-04(YYYY-MM)
freePT	The amount of PTs the employee has free for that month	Yes	Yes	5(An IEEE 754 floating point value.)(dot separated)

The csv file will follow a structure where the first row indicates the columns. The first column being email and the rest the values for the months the free pts to be assigned to. For example:

Email	2023-04	2023-05	2023-06	2023-07	2023-08	2023-09	2023-10	2023-11	2023-12
guillermo.peiro.aznar@msg.group	1	2	2	13	15	13	13	13	9.5

In the example, above we see the first column being email in lower case. Guillermo will be assigned the consecutive free PTs for the months indicated in the first row.

The csv file must be semicolon separated. The name of the file must not contain any special characters such as umlauts, please keep it simple only alphanumeric and underscores for separation.

In the first column of emails must be unique and therefor affect just one user. For example

Email	2023-04	2023-05	2023-06	2023-07	2023-08	2023-09	2023-10	2023-11	2023-12
guillermo.peiro.aznar@msg.group	1	2	2	13	15	13	13	13	9.5
guillermo.peiro.aznar@msg.group	3	4	2	5	18	20	13	13	9.5

In this case Guillermo will get the values of the second row, and the values in the first row will not be used at all.

Here an example file:

Current functionality (Deprecated)

In this section the current functionality will be described. The data model for the imported users is the one we see in Section 5.1

Current concept:

The architectural design is based on a technical user called “UserImport” in AWS. The user will have access only to a S3 bucket in the AWS account of the client. Once the users add the file to this bucket a trigger executes, and a lambda function starts the importing process. Once the data from the file is processed the user’s data is added, updated, or removed from/to DynamoDB. The next image shows an example table in csv, the format of the csv must be semicolon separated and utf-8. The columns in the first row indicate the field of the data model.

The csv file must be semicolon separated. The name of the file must not contain any special characters such as umlauts, please keep it simple only alphanumeric and underscores for separation.

Use case 1 User creation

Creation process	Priority: Low	Estimate:
The user is created when a user with this id does not exist in the database. Id is a primary key in the database and can’t be updated.
Acceptance Criteria The file is read. User that did not exist is added when newly appearing in the file.

The creation process does not include any special considerations. If the data owner (IDM) includes the data, it is allowing ProfileMap to add it the database. The user needs to be imported so it can be later registered. The user created will be the one whose id is not in the database.

For example, a user with id 1 and name Guillermo is included in the file. The database does not contain any id = 1. The new entry is created with id 1 and firstName Guillermo.

The user email should be unique with respect to the id. This means two different ids must never have the same email assigned. In the case of having two different ids with the same email, the user will get two profiles, and this will lead to errors.

For example:

ID= 1; Email guillermo.peiro.aznar@msg.group.

ID=11; Email: guillermo.peiro.aznar@msg.group.

Use case 2 User update

Update process
Upon reading the file the user is updated with the current file information if its id already exists in the database.
Acceptance Criteria The file is read. User exists and information is correctly changed.

A user will be updated if an existing id or msgId for this user are given in their corresponding columns.

Example:

User with first Name Guillermo appears in the file with id 1. Next time we import the document, Guillermo will take the information where id = 1 is specified. If for example Guillermo should change his firstName to Wilhelm, the file should contain id =1 firstName = Wilhelm.

The special considerations to do when updating a user:

1. The user id cannot be updated because it is the primary key in the database. This is the same id should always be used to reference a specific person in the real world.
2. If the user business unit/cost center is updated, the user loses all the management roles.

Use case 3 User deletion

Update process	Priority: Low	Estimate:
Upon reading the file the user information indicating the user must be removed. If a leavingDate is indicated and in the past.
Acceptance Criteria The file is read. User with a leavingDate in the past is deleted.

Every time the pipeline runs it will check if a user is to be delete depending on the data deletedAt and if delete is set to true, if so, the API will be called to delete the user.

For example, the user with id 1 and firstName Guillermo appears in the file with specified living data 2023.01.01.(YYYY-MM-DD) the user will be removed on the date or any other date in the future when the import is executed.