will database design improve sql queries performance?

Well, I get a number of questions on the relationship between learning and practicing better relational database design principles and writing better, faster or more efficient Sql queries.

In this post, I will answer all these questions or the variations of them that focus on the necessity of learning database design when all you want to do is to write sql queries!

Here are some of the variations of this question:

1. Do I need to master relational database design in other to write better Sql queries?

2. Is knowledge of normalization, relational integrity and other database development practices, relevant to writing sql queries?

3. Why do I need to master entity-relationship concepts or relational database design when all I want to do is write INNER JOINS, OUTER JOINS AND/OR SUBQUERIES?

Here is my answer to these questions asked by aspiring database designers, report writers or sql query writers:

• Efficient Sql Queries Depends On Database Index Design

  One of the best practices adopted by experienced sql query writers is adding the right types of indexes to their table columns especially those used in JOINS or in WHERE Clauses or other Sql Functions (Group BY, etc.).

  Guess what? Those indexes are added as part of the database design, database normalization or sql query performance tuning process.

  In other words, learning database design will lead to writing or designing better table indexes which will lead to better sql query performance.

• Database Normalization Reduces Data Redundancy & Inconsistency

  A lot of time can be wasted trying to get accurate, consistent, reliable reports by writing long, really cumbersome sql queries when there is redundant, inconsistent data in the underlying database.

  The real problem is that whenever you have poor normalization or database design, the resulting data redundancy or data inconsistency will make it difficult for a sql query writer to get accurate, consistent, explainable or understandable results!

  The reason is that whenever your data is stored in multiple tables because of poor normalization, your sql reports will return inconsistent results because each table contains a different set of data.

  This means that your manager or customer will not be able to get an accurate picture of the data because there is no way to verify which of the tables is right or which is wrong!

  So, if you want accurate, consistent, reliable sql reports, make sure that you are following the best normalization practices …

• Writing Overly Complex Sql Queries Because of Poor Normalization

  A heatlhcare services organization hired our company for a database consulting project because their database analysts were getting over-whelmed with providing, accurate, fast, reliable Sql-based reports to medical doctors in their organization!

  After our investigation, we found a poorly normalized database with indexes on text columns, manually generated primary keys, multi-valued domains and other bad database design practices.

  This healthcare services provider, wasted so much man-hours trying to get consistent, accurate, reliable report for their medical doctors because the results varied depending on who wrote the report or which table was joined to get the information!

  This poor database design was also causing their database analysts to write overly complicated, time consuming, error prone or complex sql queries for common, every day tasks!

  Since these database analysts were joining patient information tables on text columns, their sql queries were guaranteed to under-perform.

  Since the columns where multi-valued, sub-queries had to be used to parse the values of table columns and since the tables did not accurately model the entities used in the business or their relationships, the sql queries where awkward and non-intuitive at best!

  The solution was not to spend more time re-writing their sql queries but to normalize and redesign their database, recreate the indexes and then redesign the sql queries from scratch!

If you ever find yourself in a situation where you are bending over backwards in your work, trying to make your sql queries return the results that you want, make sure that your database design models your business process and that it follows best practices!

The job of a Sql Query Writer does not start with just writing efficient sql queries but with learning and practicing database design or normalization best practices.

Only after you ensure that your database design follows generally accepted best practices, are you empowered to write intuitive, high-performing, reliable, consistent or efficient Sql queries or reports.

What Is The Toughest Challenge Facing Your Career?

Visit the Shopping Cart, Click Here

ABOUT TRAINING & COACHING COURSES

• Business Analyst Boot Camp - Online, Web Based, Instructor Led, Business Analyst Video Training


• SQL Boot Camp - Online, Web Based, Instructor Led, Hands On, SQL + RDBMS Video Training


• Software Developer Boot Camp - Online, Web Based, Instructor Led, Software Developer Video Training


• Data Analysts Boot Camp - Online, Web Based, Instructor Led, Hands On, Data Analyst Video Training


• Get Hired Fast - Online, Web Based, Resume + Job Search + Interview Video Training


• Become A Leader Coaching - Develop hands-on experience, gain domain expertise, solve industry problems, Stand Up and Be Noticed, market your career using PULL instead of PUSH techniques!

Visit our Shopping Cart

A Step by Step Guide

While a fair amount of the work that goes into creating, maintaining or optimizing databases is hidden from decision makers, the work that goes into creating reports is immediately visible to management and so, must be given a good deal of attention.

This post is written as a guide for creating SQL based reports. It helps ensure that high quality, valid, reliable reports are created by report writers.

After reading this post, you would have a formal process for creating SQL reports and a step by step guide for reporting on data contained in AS400, Oracle, SQL Server, Microsoft Access or some other database platform.

A Step By Step Guide For Creating SQL Reports

Here is a step by step process for creating SQL Reports …

• Step One: Gather and Document the Requirements for your Report

• Step Two: Find out where you will be pulling your data from (AS400, Oracle, SQL Server, etc)

• Step Three: Write the SQL Queries for extracting your data from the database

• Step Four: Present and format your SQL Results in a Reporting Tool

• Step Five: Distribute your SQL Based Reports to the end users

Gather Or Document Requirements

Considering the complexity of reporting requirements, you may need to first document the requirements of your project before you start developing the reports.

Writing a requirements document would help you ensure that you understand your users, their reporting needs and the operational or business environment.

A requirements document will also help validate the robustness of your reporting solution upfront … before you start developing the reports.

How To Access Reporting Data From Heterogeneous Databases?

Many organizations store there data in more than one type of database which poses a challenge from a skills stand-point.

If your organization uses AS400, Oracle and SQL Server to store data and you’ve been asked to write a report that draws from all of them, you may find that challenging because each type of database uses a different flavor of SQL.

If this is your situation, then you need to start with the database that you’re the most familiar with (Microsoft Access, SQL Server, etc.) and see if you can connect to all the other databases from that.

Note that many database systems like SQL Server or MS Access offer you the ability to access the data contained in other types of databases easily.

How To Design Or Format Reports?

If you are asked to design a SQL based report that presents data in tables or charts with formatting of page titles, page numbering, column headers, sub-totals, totals, page breaks or more … here is what you can do

• You can choose a reporting tool like Crystal Reports or SQL Server Reporting Services or Microsoft Access.

  Out of these three, Microsoft Access Reports may be the easiest to learn or use in formatting or designing your reports.

• You can develop or code your reports as a web-page. Note that coding or developing your report using a web-development language like PHP or ASP.NET is a flexible but skill intensive approach to creating reports.

  Using a web development language like ASP.NET or PHP, you can execute SQL statements on a database and then return the results formatted into tables, columns, charts, etc, on your web-page.

  You can also paginate the results or add advanced searching or filtering options. When you write all the code yourself, you have complete control over what you can do or create with your SQL reports.

How To Deliver Or Distribute Reports?

Depending on the organization you work for, you may have to distribute your reports based on the preferences of your users.

For example, while some business users expect their reports to be printed and placed on their desk, others may prefer that their reports be distributed by e-mail.

You can choose between the several possible types of report delivery options including PDF, Microsoft Excel or even deliver your report as a HTML web page.

How To Schedule Reports?

Finally, you also have to decide what reporting frequency will be most appropriate for your situation (daily, weekly, monthly, end of year), etc.

Some reports take a while to prepare or consume a fair amount of server resources, so they need to be done nightly or on weekends.

Others like tax or payroll reports may need to be prepared weekly or monthly.

The takeaway is that you need to be mindful of each type of report and verify when or how often you need to prepare it.

So, now you have a set of steps for preparing reports. Start by gathering your reporting requirements. Next, determine the sources of data that will be presented on the report. Then create the reports using SQL. Finally, format and distribute the reports to the appropriate users.

What Is The Toughest Challenge Facing Your Career?

Visit the Shopping Cart, Click Here

ABOUT TRAINING & COACHING COURSES

• Business Analyst Boot Camp - Online, Web Based, Instructor Led, Business Analyst Video Training


• SQL Boot Camp - Online, Web Based, Instructor Led, Hands On, SQL + RDBMS Video Training


• Software Developer Boot Camp - Online, Web Based, Instructor Led, Software Developer Video Training


• Data Analysts Boot Camp - Online, Web Based, Instructor Led, Hands On, Data Analyst Video Training


• Get Hired Fast - Online, Web Based, Resume + Job Search + Interview Video Training


• Become A Leader Coaching - Develop hands-on experience, gain domain expertise, solve industry problems, Stand Up and Be Noticed, market your career using PULL instead of PUSH techniques!

Visit our Shopping Cart

database careers in demand

The 7 Top, In-Demand Database Careers

Database skills are foundational to jobs in the IT Industry.

By that, I mean whether you’re a business analyst, software tester, project manager, software developer or even a web designer, you’ll be using relational database and sql query writing skills at some time in your career.

Database or SQL skills are also important because they position you for multiple careers paths ranging from that of the database developer to the data mining professional.

Career Paths for Database Professionals

Here is a comprehensive list of database career paths and their respective roles, duties, responsibilities and skill sets.

1. Relational Database Developer Career

   This is one of the foundational database careers in the sense that you can hardly qualify for other database career path without a strong relational database design & development foundation.

   Database developers are responsible for designing, architecting or developing relational databases using a database development tool like Microsoft Access, SQL Server, MySQL or Oracle.

2. SQL Developer / Query Writer Career

   This is the another foundational database career in the sense that you will need strong SQL query writing skills regardless of your role, job or designation in the IT Industry.

   SQL Developers or Query Writers are typically responsible for updating, deleting, creating or manipulating the data contained in relational databases using a database programming or query language known as SQL.

   There are several dialects of SQL marketed by database vendors and each dialect is executed in the database using an administrative front-end tool like SQL Server’s T-SQL Management Studio.

3. Database Administrator (DBA) Careers

   DBAs are responsible for keeping the database up and running for the company’s operations.

   DBAs are also responsible for installing, configuring and maintaining the entire database infrastructure or platform in their organizations.

   In addition, DBAs secure their databases from unauthorized access, manage the storage available on databases and optimize their performance.

   DBAs are found in organizations that store and maintain their own data in-house using relational database management systems like SQL Server, Oracle and MySQL.

4. The Report Writer Career

   Report Writer Career paths are equally important in the database industry.

   Report Writers need to have strong relational database and SQL query writing or development skills.

   The Report Writer’s responsibility includes designing, developing, scheduling and distributing database reports.

   Report Writers utilize tools like Crystal Report Writer, Business Objects, Microsoft Access or SQL Server Reporting Services for their work.

   Report Writers also tend to spend a good amount of time collecting requirements or presenting their reports to stakeholders, executives, management, business analysts or software developers.

5. Data Analyst Career

   The Data Analyst career is frequently confused for a purely statistical career path.

   Data Analysis roles actually utilize SQL query writing, relational database development and statistical data analysis skills.

   In the IT industry, the information accessed and analyzed by data analysts resides in relational databases. So, data analysts have to also master the relational database and sql query writing path of their careers as well … and not just the statistical skills.

   A lot of people struggle with the distinctions between purely statistical careers and the database analyst careers … don’t be one of those!

6. The Data Warehousing / Business Intelligence (BI) Career

   BI and Data Warehousing careers require skillsets that are almost opposite to that of the relational database developer.

   While traditional database developers focus on their normalization skills, data warehouse / BI professionals do the exact opposit by developing enterprise, de-normalized databases.

   Another key difference is that while traditional database developers typically create Online Transaction Processing (OLTP) databases, data warehousing / BI professionals focus on building Online Analytical Processing (OLAP) databases.

   The BI / Data Warehousing professional’s responsibilities also include creating and delivering multi-dimensional reporting tools using Business Intelligence tools like Oracle’s Business Intelligence (BI) or Microsoft’s SQL Server Buisness Intelligence platform.

   From a responsibility standpoint, the data warehousing / BI professional is responsible for aggregating the data from multiple corporate databases (developed by database developers) into enterprise, corporate databases.

7. The Data Mining Career

   Data Mining careers require a rigorous background, education or training in specialized computer-based data mining algorithms or techniques.

   Data Mining professionals run algorithms like Neural Networks, Classification, Regression, Segmenattion, Association or Sequence Analysis algorithms on large databases.

   Traditionally, data mining professionals had to have a strong background in mathematics, statistics and computer science because they often had to create data mining algorithms from scratch!

   But these days, vendors like Microsoft are placing analytics and even data mining within the hands of every motivated database professional through out-of-the-box data mining algorithms like those of SQL Server’s Data Mining Suite.

The foundational skills required for database careers are taught in the SQL Boot Camp and the additional statistical data analysis requirements for data analysts taught in the Data Analysts Boot Camp.

If you are interested in gaining one of the most verstaile IT career skills, you should register for either the SQL Boot Camp or the Data Analysts Boot Camp (which includes all the courses in the SQL Boot Camp and additional statistical, data analysis training.)

The only thing left … is to take your SQL / Data Analysts training knowing that relational database development, SQL query writing and statistical data analysis skills will help advance your future for a long time!

What Is The Toughest Challenge Facing Your Career?

Visit the Shopping Cart, Click Here

ABOUT TRAINING & COACHING COURSES

• Business Analyst Boot Camp - Online, Web Based, Instructor Led, Business Analyst Video Training


• SQL Boot Camp - Online, Web Based, Instructor Led, Hands On, SQL + RDBMS Video Training


• Software Developer Boot Camp - Online, Web Based, Instructor Led, Software Developer Video Training


• Data Analysts Boot Camp - Online, Web Based, Instructor Led, Hands On, Data Analyst Video Training


• Get Hired Fast - Online, Web Based, Resume + Job Search + Interview Video Training


• Become A Leader Coaching - Develop hands-on experience, gain domain expertise, solve industry problems, Stand Up and Be Noticed, market your career using PULL instead of PUSH techniques!

Visit our Shopping Cart

Ask IT Career Coach

Can one make a career or earn a living out of just writing SQL Reports?

Considering that there’s more to the SQL Report Writer’s job than just writing reports, the answer is yes!

The following question was posted by a report writer who is trying to get a handle on the duties, roles, responsibilities or functions of a SQL Report Writer.

Can I Make A Career Out Of SQL Report Writing?

Can one make a career out of just writing SQL Reports?

If so, do you think corporations are getting lazy by overpaying SQL Report Writers $65K+ just for writing Reports.

I develop database documents, flow charts, append revisions.. When approved i create the sql, the error checking, import required data for setup… Move on to programming … building database connections, run load performance tools (custom built).. modify tables as needed, modify documentation as needed.. bench mark the sites and move on to the next…

I failed to mention the part where i build custom reports based on the clients needs, exporting to csv/excel/data dumps for other programs/complex dump algorythms

Does the SQL Report Writer have to care about the semantics of database design or the value of the data stored?

The Job Of A SQL Report Writer

The SQL Report Writer’s role is often misunderstood as just writing reports.

The reality is that a SQL Report Writer’s job is often more involved!

The Report Writer has to know the value of the data, where it is coming from, how to get at or extract it, how to clean it and verify that it is accurate and how to present information correctly using charts, graphs, aggregates, statistical computations, etc.

1. The Report Writer’s duties includes optimizing or fine tuning slow running reports. The performance of a report may degrade over time as additional data is loaded into tables or as more tables are joined to get at the required data.

   A report may degrade in performance over a period of time or run slowly because the joins are inefficient or because the sql statements need to be optimized or because the data structures (indexes) used in the tables need to be optimized or for other reasons.

   Whatever the situation, the Report Writer is responsible for monitoring the performance of reports and taking corrective action to optimize or improve the performance of the reports when necessary.

2. The Report Writer’s job role includes reconciling discrepancies between the expectations of data managers and information presented in their reports. A report writer has to spend part of their time validating or proving to decision managers that the data presented on their results are valid or accurate.

3. The Enterprise Report Writer is responsible for writing the reports needed for different teams, managers, decision makers or departments.

   Some of the reports may use computed or aggregated columns (averages, sums or statistical computations) and the report writer has to write subqueries, nested queries and complex sql statements to present the required data.

4. The Report Writer may have to access, import, extract, transfer or load data from heterogeneous data sources (Oracle, AS/400, SQL SERVER, TEXT FILES, Microsoft Access).

   As enterprises grow, they collect data across a network of legacy mainframe applications, geographic boundaries, time zones, and corporate divisions, data warehouses and databases.

5. The Report Writer’s job description includes presenting timely, up to date or current data. This is especially important for time sensitive reports or nightly reports that need to run successfully before the day shift starts or reports that are compiled at the end of a quarter or business cycle.

   Time sensitive reports include monthly statements (bank statements) or reports that need to be printed or mailed by a certain date or reports that need to be available early in the morning for a trading / securities desk or financial / investment team.

Yes, there are lot of reasons why a company will hire dedicated SQL Report Writers and Pay them Well

This post answers a question about “Making A Career Out Of SQL Report Writing”!

If you have any Question about your career, Post IT as a comment on this page and I will be sure to answer it for you!

What Is The Toughest Challenge Facing Your Career?

Visit the Shopping Cart, Click Here

ABOUT TRAINING & COACHING COURSES

• Business Analyst Boot Camp - Online, Web Based, Instructor Led, Business Analyst Video Training


• SQL Boot Camp - Online, Web Based, Instructor Led, Hands On, SQL + RDBMS Video Training


• Software Developer Boot Camp - Online, Web Based, Instructor Led, Software Developer Video Training


• Data Analysts Boot Camp - Online, Web Based, Instructor Led, Hands On, Data Analyst Video Training


• Get Hired Fast - Online, Web Based, Resume + Job Search + Interview Video Training


• Become A Leader Coaching - Develop hands-on experience, gain domain expertise, solve industry problems, Stand Up and Be Noticed, market your career using PULL instead of PUSH techniques!

Visit our Shopping Cart

The report writer as a key member of the database, sql or data analytics team is responsible for creating, validating, distributing and documenting the reports required by an organization.

Report Writers work with various types of relational databases, sql query writing and report writing tools because they need to retrieve, aggregate and present data from different databases including Oracle, SQL Server or ODBC Compliant Databases.

Since it is not practical or feasible for Report Writers to learn or master all the available database reporting tools, they specialize in a few and then learn any additional reporting tool come along on the job!

Report Writers can port report writing skills from one Reporting Platform to another because Reporting tools are based on these major components:

1. Report Writing Databases:  Report Writing Databases act as repositories of the company’s reporting data. The data needed for report writing projects may be stored in data warehouses, proprietary databases, relational databases or ODBC compliant databases.

   Report Writers usually have to create Reports based on data distributed across multiple databases or tables. Because writing all the SQL queries, sub queries, joins, aggregate queries, rollups, etc. can get quite involved, they need to have strong SQL Skills, database skills and data analysis skills.

2. SQL Queries: Report Writers have to identify the tables that contain the required data based on the reporting requirements handed to them.

   After identifying Reporting Requirements, they may have to write complex SQL queries to extract the information, trends, or patterns contained in the data. The final query is often saved for re-use as a stored procedure.

Report Writer Duties

Report Writers work closely with end-users, stakeholders or management to gather and document reporting requirements and to test or validate their reports after.

Here are some of the roles and responsibilties of Report Writers:

1. SQL Data Retrieval: Report Writers need to write SQL code for extracting, summarizing, aggregating or presenting data from one or more database systems.

2. Data Validation: Report Writers need to validate the results of their SQL Queries with the end users because of discrepancies that may exist between what the reports say and what the users expect!

3. Data Presentation: Report Writers present their results in the form of tables, pie charts, bar charts and other plots or diagrams. The data presentation duties of a Report Writer is facilitated with Reporting Tools like SQL Server Reporting Services, Crystal Reports or Microsoft Access Reports.

Report Writer Careers

Report Writers use tools that they are familiar with to retrieve data from different databases including Oracle, SQL Server or ODBC Compliant Databases.

It is important to note that a Report Writer who is skilled in one tool like SQL Server Reporting Services and strong in the essentials of relational database design, development and sql query writing can connect to most databases (Oracle, Sybase, Microsoft Access, AS 400, JD Edwards), retrieve the data to SQL Server and then write the Reports.

Since it is not practical or feasible for Report Writers to learn or master all the database reporting tools out their, all that they need to do, is specialize in a few reporting tools and then learn any new reporting tools for their specific job situation.

A skilled Report Writers can be placed in a variety of positions including:

1. Sql Server Reporting Services Report Writer

2. Ms Access Report Writer

3. Crystal Report Writer

4. PL SQL/Oracle Forms/Report Writer

5. SQL Report Writer

6. Hyperion Report Writer

7. Cognos Report Writer

8. IBM COBOL Report Writer

9. Visual FoxPro Report Writer

10. Database Report Writer

11. AS400 Report Writer

Report Writer Training – How to Become a SQL Report Writer

Report Writers may choose only one of these two training options, based on their career aspirations or job requirements.

1. SQL Boot Camp: The SQL Boot Camp provides you with all the database development and sql query writing skills needed for Report Writing jobs.

2. Data Analyst Boot Camp: The Data Analyst Boot Camp provides you with all the database development, sql query writing, data analysis and statistical analysis skills needed for Report Writing jobs.

In this article, I discussed the skill requirements, duties, roles, responsibilities and career paths available to Report Writers.

If you have any question about becoming a Report Writer, go ahead and post it as a comment at the end of this post and I will be glad to answer it for you!

What Is The Toughest Challenge Facing Your Career?

Visit the Shopping Cart, Click Here

ABOUT TRAINING & COACHING COURSES

• Business Analyst Boot Camp - Online, Web Based, Instructor Led, Business Analyst Video Training


• SQL Boot Camp - Online, Web Based, Instructor Led, Hands On, SQL + RDBMS Video Training


• Software Developer Boot Camp - Online, Web Based, Instructor Led, Software Developer Video Training


• Data Analysts Boot Camp - Online, Web Based, Instructor Led, Hands On, Data Analyst Video Training


• Get Hired Fast - Online, Web Based, Resume + Job Search + Interview Video Training


• Become A Leader Coaching - Develop hands-on experience, gain domain expertise, solve industry problems, Stand Up and Be Noticed, market your career using PULL instead of PUSH techniques!

Visit our Shopping Cart