# Old manifest: 2910f7d05f74e086c097595fe8cc78ad728474e4 # New manifest: f4a4fb1871edfc75efcfa80ade602e36463909f0 # Summary of changes: # # patch doc/FAQ_DEV # from 6ea09a083520481b8896b30432a85af841655f31 # to 4b017fc05c35d88807fc61a5d12767ac5838eecb # # patch src/backend/access/transam/xlog.c # from 44af82f3c379aad3d842fc2b903d0211ecdd1e30 # to 08dcb5592d1c8a9501dd6e8845c32b96710b987c # --- doc/FAQ_DEV +++ doc/FAQ_DEV @@ -12,18 +12,18 @@ General Questions 1.1) How do I get involved in PostgreSQL development? - 1.2) How do I add a feature or fix a bug? - 1.3) How do I download/update the current source tree? - 1.4) How do I test my changes? - 1.5) What tools are available for developers? - 1.6) What books are good for developers? - 1.7) What is configure all about? - 1.8) How do I add a new port? - 1.9) Why don't you use threads/raw devices/async-I/O, ? - 1.10) How are RPM's packaged? - 1.11) How are CVS branches handled? - 1.12) Where can I get a copy of the SQL standards? + 1.9) How are RPMs packaged? + 1.10) How are CVS branches handled? + 1.11) Where can I get a copy of the SQL standards? + 1.12) Where can I get technical assistance? Technical Questions @@ -42,154 +42,144 @@ 1.1) How go I get involved in PostgreSQL development? - This was written by Lamar Owen: + 1.1.1) Where do I start? + + Download the code and have a look around. See 1.2. - 2001-06-22 - What open source development process is used by the PostgreSQL team? - - Read HACKERS for six months (or a full release cycle, whichever is - longer). Really. HACKERS _is_the process. The process is not well - documented (AFAIK -- it may be somewhere that I am not aware of) -- - and it changes continually. - What development environment (OS, system, compilers, etc) is required + Subscribe to and read the pgsql-hackers mailing list (often termed + 'hackers'). This is where the major contributors and core members of the + project discuss development. + + 1.1.2) What development environment (OS, system, compilers, etc) is required to develop code? - Developers Corner on the website has links to this information. The - distribution tarball itself includes all the extra tools and documents - that go beyond a good Unix-like development environment. In general, a - modern unix with a modern gcc, GNU make or equivalent, autoconf (of a - particular version), and good working knowledge of those tools are - required. - What areas need support? + PostgreSQL is developed mostly in the C programming language. It also + makes use of Yacc and Lex. + + The source code is targeted at most of the popular Unix platforms and + the Windows environment (XP, Windows 2000, and up). + + Most developers make use of the open source development tool chain which + you are probably familiar with if you've compiled or contributed to + open source software before. Developers using this tool chain on Windows + make use of MingW (see http://www.mingw.org/). + + 1.1.3) What areas need work? - The TODO list. + Outstanding features are detailed in the TODO list. This is located in + doc/TODO in the source distribution or at + http://developer.postgresql.org/todo.php. + + You can learn more about these features by consulting the archives, the + SQL standards and the recommend texts (see 1.6). + + 1.1.4) What do I do after choosing an item to work on? + + Send an email to pgsql-hackers with a proposal for what you want to do + (assuming your contribution is not trivial). Working in isolation is not + advisable: others may be working on the same TODO item; you may have + misunderstood the TODO item; your approach may benefit from the review of + others. + + 1.1.5) Where can I learn more about the code? + + Other than documentation in the source tree itself, you can find some + papers/presentations discussing the code at http://developers.postgresql.org + + 1.1.6) I've developed a patch, what next? + + Generate the patch in contextual diff format. If you are unfamiliar with + this, you may find the script src/tools/makediff/difforig useful. + + Ensure that your patch is generated against the most recent version of the + code. If it is a patch adding new functionality, the most recent version is + cvs HEAD; if it is a bug fix, this will be the most recently version of the + branch which suffers from the bug (for more on branches in PostgreSQL, + see 1.11). - You've made the first step, by finding and subscribing to HACKERS. - Once you find an area to look at in the TODO, and have read the - documentation on the internals, etc, then you check out a current - CVS,write what you are going to write (keeping your CVS checkout up to - date in the process), and make up a patch (as a context diff only) and - send to the PATCHES list, prefereably. + Finally, submit the patch to pgsql-patches@postgresql.org. It will be + reviewed by other contributors to the project and may be either accepted + or sent back for further work. - Discussion on the patch typically happens here. If the patch adds a - major feature, it would be a good idea to talk about it first on the - HACKERS list, in order to increase the chances of it being accepted, - as well as toavoid duplication of effort. Note that experienced - developers with a proven track record usually get the big jobs -- for - more than one reason. Also note that PostgreSQL is highly portable -- - nonportable code will likely be dismissed out of hand. - - Once your contributions get accepted, things move from there. - Typically, you would be added as a developer on the list on the - website when one of the other developers recommends it. Membership on - the steering committee is by invitation only, by the other steering - committee members, from what I have gathered watching froma distance. - - I make these statements from having watched the process for over two - years. - - To see a good example of how one goes about this, search the archives - for the name 'Tom Lane' and see what his first post consisted of, and - where he took things. In particular, note that this hasn't been _that_ - long ago -- and his bugfixing and general deep knowledge with this - codebase is legendary. Take a few days to read after him. And pay - special attention to both the sheer quantity as well as the - painstaking quality of his work. Both are in high demand. - - 1.2) How do I add a feature or fix a bug? + 1.2) How do I download/update the current source tree? - The source code is over 350,000 lines. Many fixes/features are - isolated to one specific area of the code. Others require knowledge of - much of the source. If you are confused about where to start, ask the - hackers list, and they will be glad to assess the complexity and give - pointers on where to start. - - Another thing to keep in mind is that many fixes and features can be - added with surprisingly little code. I often start by adding code, - then looking at other areas in the code where similar things are done, - and by the time I am finished, the patch is quite small and compact. - - When adding code, keep in mind that it should use the existing - facilities in the source, for performance reasons and for simplicity. - Often a review of existing code doing similar things is helpful. - - The usual process for source additions is: - * Review the TODO list. - * Discuss hackers the desirability of the fix/feature. - * How should it behave in complex circumstances? - * How should it be implemented? - * Submit the patch to the patches list. - * Answer email questions. - * Wait for the patch to be applied. - - 1.3) How do I download/update the current source tree? - There are several ways to obtain the source tree. Occasional developers can just get the most recent source tree snapshot from - ftp.postgresql.org. For regular developers, you can use CVS. CVS - allows you to download the source tree, then occasionally update your - copy of the source tree with any new changes. Using CVS, you don't - have to download the entire source each time, only the changed files. - Anonymous CVS does not allows developers to update the remote source - tree, though privileged developers can do this. There is a CVS section - (http://developer.postgresql.org/docs/postgres/cvs.html) in our - documentation that describes how to use remote CVS. You can also use - CVSup, which has similarly functionality, and is available from - ftp.postgresql.org. - - To update the source tree, there are two ways. You can generate a - patch against your current source tree, perhaps using the make_diff - tools mentioned above, and send them to the patches list. They will be - reviewed, and applied in a timely manner. If the patch is major, and - we are in beta testing, the developers may wait for the final release - before applying your patches. - - For hard-core developers, Marc(scrappy@postgresql.org) will give you a - Unix shell account on postgresql.org, so you can use CVS to update the - main source tree, or you can ftp your files into your account, patch, - and cvs install the changes directly into the source tree. - - 1.4) How do I test my changes? + ftp.postgresql.org. + + Regular developers may want to take advantage of anonymous access to + our source code management system. The source tree is currently hosted in + CVS. For details of how to obtain the source from CVS see + http://developer.postgresql.org/docs/postgres/cvs.html. + + 1.3) How do I test my changes? + + 1.3.1) Basic system testing + + The easiest way to test your code is to ensure that it builds against + the latest verion of the code and that it does not generate compiler + warnings. + + Then, perform run time testing via psql. + + 1.3.2) Regression test suite + + The next step is to test your changes against the existing regression test + suite. To do this, issue "make check" in the root directory of the source + tree. If any tests failure, investigate. + + If you've deliberately changed existing behaviour, this change may cause a + regression test failure but not any actual regression. If so, you should + also patch the regression test suite. + + 1.3.3) Other run time testing + + Some developers make use of tools such as valgrind + (http://valgrind.kde.org) for memory testing, gprof (which comes with + the GNU binutils suite) and oprofile (http://oprofile.sourceforge.net/) + for profiling and other related tools. + + 1.3.4) What about unit testing, static analysis, model checking...? + + There have been a number of discussions about other testing frameworks + and some developers are exploring these ideas. + + 1.4) What tools are available for developers? - First, use psql to make sure it is working as you expect. Then run - src/test/regress and get the output of src/test/regress/checkresults - with and without your changes, to see that your patch does not change - the regression test in unexpected ways. This practice has saved me - many times. The regression tests test the code in ways I would never - do, and has caught many bugs in my patches. By finding the problems - now, you save yourself a lot of debugging later when things are - broken, and you can't figure out when it happened. - - 1.5) What tools are available for developers? - - Aside from the User documentation mentioned in the regular FAQ, there - are several development tools available. First, all the files in the - /tools directory are designed for developers. + First, all the files in the src/tools directory are designed for developers. + RELEASE_CHANGES changes we have to make for each release - SQL_keywords standard SQL'92 keywords backend description/flowchart of the backend directories ccsym find standard defines made by your compiler + copyright fixes copyright notices entab converts tabs to spaces, used by pgindent find_static finds functions that could be made static find_typedef finds typedefs in the source code find_badmacros finds macros that use braces incorrectly + fsync a script to provide information about the cost of cache + syncing system calls make_ctags make vi 'tags' file in each directory make_diff make *.orig and diffs of source make_etags make emacs 'etags' files make_keywords make comparison of our keywords and SQL'92 make_mkid make mkid ID files - mkldexport create AIX exports file - pgindent indents C source files - pgjindent indents Java source files + pgcvslog used to generate a list of changes for each release pginclude scripts for adding/removing include files - unused_oids in pgsql/src/include/catalog + pgindent indents source files + pgtest a semi-automated build system + thread a thread testing script + In src/include/catalog: + + unused_oids a script which generates unused OIDs for use in system + catalogs + duplicate_oids finds duplicate OIDs in system catalog definitions + Let me note some of these. If you point your browser at the - file:/usr/local/src/pgsql/src/tools/backend/index.html directory, you - will see few paragraphs describing the data flow, the backend - components in a flow chart, and a description of the shared memory - area. You can click on any flowchart box to see a description. If you - then click on the directory name, you will be taken to the source + tools/backend/index.html file, you will see few paragraphs describing the + data flow, the backend components in a flow chart, and a description of the + shared memory area. You can click on any flowchart box to see a description. + If you then click on the directory name, you will be taken to the source directory, to browse the actual source code behind it. We also have several README files in some source directories to describe the function of the module. The browser will display these when you enter @@ -261,7 +251,7 @@ pgindent will the format code by specifying flags to your operating system's utility indent. This article describes the value of a - constent coding style. + consistent coding style. pgindent is run on all source files just before each beta test period. It auto-formats all source files to make them consistent. Comment @@ -276,7 +276,7 @@ is also a script called unused_oids in pgsql/src/include/catalog that shows the unused oids. - 1.6) What books are good for developers? + 1.5) What books are good for developers? I have four good books, An Introduction to Database Systems, by C.J. Date, Addison, Wesley, A Guide to the SQL Standard, by C.J. Date, et. @@ -286,7 +286,7 @@ There is also a database performance site, with a handbook on-line written by Jim Gray at http://www.benchmarkresources.com. - 1.7) What is configure all about? + 1.6) What is configure all about? The files configure and configure.in are part of the GNU autoconf package. Configure allows us to test for various capabilities of the @@ -308,7 +308,7 @@ removed, so you see only the file contained in the source distribution. - 1.8) How do I add a new port? + 1.7) How do I add a new port? There are a variety of places that need to be modified to add a new port. First, start in the src/template directory. Add an appropriate @@ -325,7 +325,7 @@ src/makefiles directory for port-specific Makefile handling. There is a backend/port directory if you need special files for your OS. - 1.9) Why don't you use threads/raw devices/async-I/O, ? There is always a temptation to use the newest operating system @@ -351,7 +351,7 @@ cautious about their adoption. The TODO list often contains links to discussions showing our reasoning in these areas. - 1.10) How are RPM's packaged? + 1.9) How are RPMs packaged? This was written by Lamar Owen: @@ -446,7 +446,7 @@ Of course, there are many projects that DO include all the files necessary to build RPMs from their Official Tarball (TM). - 1.11) How are CVS branches managed? + 1.10) How are CVS branches managed? This was written by Tom Lane: @@ -505,7 +505,7 @@ tree right away after a major release --- we wait for a dot-release or two, so that we won't have to double-patch the first wave of fixes. - 1.12) Where can I get a copy of the SQL standards? + 1.11) Where can I get a copy of the SQL standards? There are three versions of the SQL standard: SQL-92, SQL:1999, and SQL:2003. They are endorsed by ANSI and ISO. Draft versions can be @@ -522,6 +522,20 @@ * http://www.contrib.andrew.cmu.edu/~shadow/sql.html#syntax (SQL-92) * http://dbs.uni-leipzig.de/en/lokal/standards.pdf (paper) + 1.12) Where can I get technical assistance? + + Many technical questions held by those new to the code have been answered + on the pgsql-hackers mailing list -- the archives of which can be found at: + http://archives.postgresql.org/pgsql-hackers/. + + If you cannot find discussion or your particular question, feel free to + put it to the list. + + Major contributors also answer technical questions, including questions + about development of new features, on IRC at irc.freenode.net in the + #postgresql channel. + + Technical Questions 2.1) How do I efficiently access information in tables from the backend code?