Mastering Financial Pattern Recognition (Sofien Kaabar) (Z-Library)

Praise for Mastering Financial Pattern Recognition Very well written. The book makes complicated stuff an easy read and is a must if you are passionate about trading. —Saby Upadhyay, CEO, White Swan Global Markets An extremely useful book for those interested in technical analysis, especially candlestick analysis. Easy to read and follow. I highly recommend this book for beginners and veterans alike. —Sattam Al-Sabah, president, OneUp Trader Provides a wealth of information on technical analysis and candlestick charting. Having the Python code as a starting point helps the reader implement and understand things that much faster. —Timothy M. Kipper, Jr., founder, JDTK Investments A comprehensive guide for trading using candlestick pattern-based signals, with detailed illustrations of indicators, easily implementable code, and a thorough overview of building your own systematic trading strategy. —Ning Wang, quantitative investment strategies structurer, Barclays

Mastering Financial Pattern Recognition Finding and Back-Testing Candlestick Patterns with Python Sofien Kaabar

Mastering Financial Pattern Recognition by Sofien Kaabar Copyright © 2023 Sofien Kaabar. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Michelle Smith Development Editor: Corbin Collins Production Editor: Elizabeth Faerm Copyeditor: Kim Wimpsett Proofreader: Paula L. Fleming Indexer: Potomac Indexing, LLC Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea October 2022: First Edition Revision History for the First Edition

2022-10-18: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781098120474 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Mastering Financial Pattern Recognition, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-098-12047-4 [LSI]

Preface Finding patterns is the essence of wisdom. —Dennis Prager With technological progress and the decentralization of financial information, coding and automated research have become integral parts of the trading world. Anyone who masters the art of trading and coding has a tremendous edge in the markets. Trading techniques are numerous, and they can be based on many tools and concepts. For example, fundamental traders rely on economic and political analyses to derive a long-term view on different assets, while technical traders rely on more quantitative measures and a few psychological concepts to forecast the next likely moves of the markets. Therefore, we can say that on a high level there exist two types of analyses, fundamental and technical. This book will present in detail a field in technical analysis called candlestick pattern recognition. Why This Book? I have spent my career researching trading strategies, patterns, and anything related to the financial world. I have developed a special passion for patterns, and specifically candlestick patterns, due to their widespread adoption but also their interesting performance results. Moreover, throughout the years, I have discovered a few candlestick patterns that I believe can at least rival the classical patterns. This brings us to the purpose of writing the book: I am aiming to present the totality of candlestick patterns, including my personal ones, and how to code a system that back- tests them across a wide variety of markets. Machines can perform pattern recognition and detection better than humans because of their objectivity. Therefore, I have dedicated the first chapters of

the book to creating the structure of a candlestick pattern recognition algorithm before moving on to dig deep into patterns and strategies in the later chapters. This means that the first skill you will learn is how to automate the data import process in Python. There are many classical candlestick patterns, and it is everyone’s duty to test them to see whether they actually are predictive. After all, if we use these patterns to forecast the markets, we should have objective results that prove they are indeed value-adders. We will get such results and interpret them, just as I do with the candlestick patterns that I have discovered over the years. We will also see the advantages and limitations of every pattern. When we do find the good patterns that help with the predictive task, you should insert them in the overall trading framework, which includes other tools and a risk management system. You will learn how to code technical indicators and combine them with candlestick patterns to create trading signals. Finally, you will back-test these signals, and you will be able to optimize the parameters so that you get a good full-scale pattern recognition strategy. Hence, the utility of this book is to show you how to automate your research by letting the algorithms you create evaluate the different candlestick patterns. Finally, you will learn how to determine your strategy, which will use the patterns and combine with other technical indicators. Target Audience This book is suited to aspiring students, academics, curious minds, and finance practitioners who are interested in candlestick pattern recognition and its applications in finance. You will benefit from this book if you are interested not only in using Python but also in developing strategies and technical indicators. The book assumes you have basic background knowledge in both Python programming (professional Python users will find the code very straightforward) and financial trading. I take a clear and simple approach

that focuses on the key concepts so that you understand the purpose of every idea. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values determined by context. TIP This element signifies a tip or suggestion. NOTE This element signifies a general note.

WARNING This element indicates a warning or caution. Using Code Examples Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/sofienkaabar/mastering-financial-pattern- recognition. If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Mastering Financial Pattern Recognition by Sofien Kaabar (O’Reilly). Copyright 2023 Sofien Kaabar, 978-1-098-12047-4.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. O’Reilly Online Learning

NOTE For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit https://oreilly.com. How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/mstrg- finan-pttrn-recog. Email bookquestions@oreilly.com to comment or ask technical questions about this book.

For news and information about our books and courses, visit https://oreilly.com. Find us on LinkedIn: https://linkedin.com/company/oreilly-media. Follow us on Twitter: https://twitter.com/oreillymedia. Watch us on YouTube: https://youtube.com/oreillymedia. Acknowledgments Nothing would be the same without the support of my parents, which is why I can’t help but acknowledge their direct and indirect impact on the book. I would also like to mention that none of this would be possible without the patience of my wife, Charline, who tolerated my late-night writing. I have nothing but gratitude toward her. I would also like to acknowledge the debt I owe to the editors, Michelle Smith and Corbin Collins, as well as production editor Elizabeth Faerm, for their continued support and the amazing job they do and also for their patience. Similarly, I would like to thank every person involved at O’Reilly Media. Additionally, my special thanks go to the great technical reviewers, Ning Wang, Timothy Kipper, and Kushan Vora, for their immense contributions. They have had a sizable impact on making this book readable, useful, and straightforward. I could not ask for better people to review my book. Finally, I am deeply grateful to you, the reader, for investing your time into reading my work and for placing your trust in my research. I hope you find it useful.

Chapter 1. Importing and Processing Financial Data in Python This chapter is dedicated to laying the foundation needed to analyze financial data through coding. This requires some preparation, such as downloading the right software and creating an algorithm that fetches historical data automatically. By the end of the chapter, you should know how to automatically import historical financial data using Python, a skill that should save you time. So let’s get started. Installing the Environment The first step is to prepare the environment and everything else necessary for the success of the algorithms. For this, you need two programs: A Python interpreter that you use to write and execute code Charting and financial software that you use as a database Let’s start with the Python interpreter. I use a software called SPYDER. Some people may be more familiar with other software such as Jupyter and PyCharm, but the process is the same. You can download SPYDER from the official website or, even better, download it as part of a bigger package called Anaconda, which facilitates installation and offers more tools. Note that it is open source, free-to-use software. SPYDER’s interface is split into three windows, as you can see in Figure 1- 1. The window on the left is used to write the code that is later executed

(the algorithm is told to run and apply the code). Typically, you will see multiple lines of code in that area. The window on the upper right is the variable explorer. Every time a variable is stored, you can see it there. The window on the lower right is the console that shows the result of the code, whether it is an error or an output. Figure 1-1. SPYDER’s interface The types of data that you can define and use in the code are classified into several categories: Integers These are whole numbers, which can be either positive or negative. Examples are −8 and 745. They are, however, limited to between −2147483648 and 2147483647. Any number falling outside this range is considered a different data type called a long. The difference between

data types has to do with memory. Integers are 32 bits in width, whereas longs are 64 bits in width. Floats These are real numbers with decimal points such as 18.54 and 311.52. Strings These are words stored in a variable. More scientifically, they are a set of structured characters (text). In Python, you write strings between single or double quotes. In line 1 of the code in Figure 1-1, I have defined a variable called age and set it to 13. When you run this code, you should see the creation of age in the variable explorer with type int (integer) and a value of 13. In line 3, I have executed the code that defines the height variable set to 184.50 (therefore, a float data type). Notice that next to the definition of the variable, I have written the phrase in centimeters, preceded by a hashmark. This is called a comment. Comments are very important in Python for explaining the code. Therefore, anything preceded by # will not be executed. In the variable explorer, you can see the height variable with the float type. Line 5 defines a string, which in the variable explorer is shown as str type (string). In the console, you can see that the code has been successfully executed because there are no errors, which would be shown in red. The next step in preparing the environment is to install the charting software that allows you to import historical data into SPYDER. Throughout the book, I use MetaTrader 5, a benchmark charting program used by many traders around the globe. Follow these steps: 1. Download SPYDER and familiarize yourself with how it works. 2. Download the MetaTrader 5 software.

3. Use SPYDER to import historical prices from MetaTrader 5. From the official website, download and install MetaTrader 5. You need to create a demo account, which is simply a virtual account with imaginary money. The word demo does not refer to a limited duration of use but to the fact that it is not using real money. To open an account, select File > Open an Account, choose MetaQuotes Software Corp, and then click Next. Next, choose the first option to open a demo account; this will let you trade virtual money. Finally, enter some basic information such as name, email, and account type. You will not receive a verification request or any type of confirmation as the demo should launch directly, allowing you to see the charts. Figure 1-2 shows the platform’s interface. By default, MetaTrader 5 does not show all the markets it covers, so you need to make them accessible for import and visualization if necessary. Click View, click Market Watch, and then right-click any of the symbols shown in the new tab and choose Show All. This way you can see the extended list with more markets.

Figure 1-2. MetaTrader 5’s interface Creating the Importing Algorithm  Being able to automatically summon historical data of any time frame is a wonderful time-saver as it allows you to focus on research and analysis instead of wasting valuable time acquiring and cleaning the data. Let’s create a set of functions that import the historical data of a selected asset almost instantaneously. Before proceeding to the coding part, you need to install the MetaTrader 5 Python integration library so you can use it later in SPYDER. This is easy and requires one step. Open the Anaconda prompt and type in pip install Metatrader5, as shown in Figure 1-3.

Figure 1-3. Anaconda prompt showing the command to install the MetaTrader 5 library Installation is the bridge that allows you to use Python libraries designed for MetaTrader 5 in the interpreter. The following code block uses the import built-in statement, which calls for internal (self-created) or external (created by third parties) libraries. A library is a store of functions, and thus, you need to import the libraries that are pertinent to what you want to do. For demonstration purposes, import the following modules, packages, and libraries: import datetime import pytz import pandas as pd import MetaTrader5 as mt5 import numpy as np  The datetime module gives tools for manipulating the dates and times, the pytz library offers cross-platform time zone calculations that are needed for the import, and the pandas and numpy libraries are used for data manipulation and analysis. NOTE You mainly use numpy for most calculations and data manipulation. The MetaTrader 5 library imports the functions relating to the software’s module and is the key library that will allow you to import the financial historical data. Note that the three last lines of code contain the as statement. This is used to give a custom name to the library when you want to use it frequently and

save writing space. In other words, Python recognizes the MetaTrader 5 library as mt5 from now on. NOTE Modules are files that contain functions and variables. A package is a collection of modules; it needs to have an init.py file. A library is simply a collection of packages. Executing the import statements means that Python now recognizes the functions inside them and will allow you to use them in future code if you decide to call them. You must run them every time you open a new session, which is why the import statements are usually found at the beginning of the code. The next step is to create the universe of the time frames that you will be able to import. Even though I will be showing you how to analyze and back-test hourly data, you can define a wider universe, as shown in the following code snippet: frame_M15 = mt5.TIMEFRAME_M15 # 15-minute time frameframe_M30 = mt5.TIMEFRAME_M30 # 30-minute time frame frame_H1 = mt5.TIMEFRAME_H1 # Hourly time frame frame_H4 = mt5.TIMEFRAME_H4 # 4-hour time frame frame_D1 = mt5.TIMEFRAME_D1 # Daily time frame frame_W1 = mt5.TIMEFRAME_W1 # Weekly time frame frame_M1 = mt5.TIMEFRAME_MN1 # Monthly time frame A time frame is the frequency with which you record the prices. With hourly data, you will record the last price printed every hour. This means that in a day, you can have up to 24 hourly prices. This allows you to see the intraday evolution of the price. However, the close price is just one of the things that you want to import. During a time period (whether hourly or daily), you will see the following: The first price of the time period, which is called the open price.

The highest price printed during the time period, which is called the high price.  The lowest price printed during the time period, which is called the low price. The last price seen before starting a new time period, which is referred to as the close price. Altogether these are called the OHLC data, which are generally ordered as written. The following code defines the current time, which is used so that the algorithm has a reference point when importing the data. Basically, you are creating a variable that stores the current time and date: now = datetime.datetime.now() Let’s now proceed to defining the universe of the assets you want to back- test. I do a mix of four asset classes: currencies, cryptocurrencies, commodities, and equity indices: Currencies (also known as forex, an abbreviation for foreign exchange market) form the biggest financial market in terms of daily volume. Currencies are quoted in pairs, meaning that you cannot just buy USD in absolute terms; you have to buy it using another currency. Therefore, the EURUSD pair refers to the price of 1 EUR in terms of USD. The back-testing universe comprises EURUSD, USDCHF, GBPUSD, and USDCAD. NOTE USD is an abbreviation for the United States dollar, EUR is an abbreviation for the euro currency, CHF is an abbreviation for the Swiss franc, GBP is an abbreviation for Great Britain’s pound, and CAD is an abbreviation for the Canadian dollar. 1

Cryptocurrencies (also known as cryptos) are a new, disruptive asset class characterized by severe volatility. The most well-known cryptocurrency is Bitcoin, followed by Ethereum. Note that both are expressed in terms of USD; this is why they are labeled as BTCUSD and ETHUSD. NOTE Notice that Bitcoin (BTC) and Ethereum (ETH) are quoted versus USD. They are generally considered the most liquid cryptocurrency pairs. Commodities are physical assets such as gold, silver, and copper. They are divided into many categories such as energy (crude oil, Brent oil, etc.) and industrial metals (copper, zinc, etc.). In the universe of assets, I stick to gold and silver. Equity indices are weighted calculations of a select basket of a country’s stocks. They are used to analyze the overall stock market health of a nation. In this book, I cover the S&P 500, a proxy for US stocks, and the FTSE 100, a proxy for UK stocks: assets = ['EURUSD', 'USDCHF', 'GBPUSD', 'USDCAD', 'BTCUSD',   'ETHUSD', 'XAUUSD', 'XAGUSD', 'SP500m', 'UK100'] Now that you have your time and asset variables ready, all you need is to create the structure of the importing algorithm. The get_quotes() function does this: def get_quotes(time_frame, year = 2005, month = 1, day = 1,   asset = "EURUSD"):              if not mt5.initialize():                  print("initialize() failed, error code =", mt5.last_error())