Re: On How To Shorten the Steep Learning Curve Towards PG Hacking...

Thanks Amit for taking your time and pointing to some useful stuff on the
Internals of PostgreSQL.

One thing I have learned is that PG community is not as hostile/harsh as I
imagined to newbies. Rather, its the reverse.
I am feeling at home here.

Amit, would you please help out on how to apply some patches in PG source
code. For example, there are two patches attached here: one on
CORRESPONDING CLAUSE and one on MERGE SQL Standard.

There are some errors saying Hunk failed(src/backend/parser/gram.y.rej).

postgresql-9.6.2$ patch --dry-run -p1 < corresponding_clause_v12.patch
patching file doc/src/sgml/queries.sgml
Hunk #1 succeeded at 1603 (offset 2 lines).
Hunk #2 succeeded at 1622 (offset 2 lines).
Hunk #3 succeeded at 1664 (offset 2 lines).
patching file doc/src/sgml/sql.sgml
patching file src/backend/nodes/copyfuncs.c
Hunk #1 succeeded at 2807 (offset -188 lines).
Hunk #2 succeeded at 2823 (offset -188 lines).
Hunk #3 succeeded at 4251 (offset -340 lines).
patching file src/backend/nodes/equalfuncs.c
Hunk #1 succeeded at 995 (offset -55 lines).
Hunk #2 succeeded at 1009 (offset -55 lines).
Hunk #3 succeeded at 2708 (offset -230 lines).
patching file src/backend/nodes/nodeFuncs.c
Hunk #1 succeeded at 3384 (offset -60 lines).
patching file src/backend/nodes/outfuncs.c
Hunk #1 succeeded at 2500 (offset -164 lines).
Hunk #2 succeeded at 2793 (offset -179 lines).
Hunk #3 succeeded at 2967 (offset -184 lines).
patching file src/backend/nodes/readfuncs.c
Hunk #1 succeeded at 414 (offset -2 lines).
patching file src/backend/nodes/value.c
patching file src/backend/optimizer/prep/prepunion.c
Hunk #1 succeeded at 92 (offset 1 line).
Hunk #2 succeeded at 112 (offset 1 line).
Hunk #3 succeeded at 190 (offset 1 line).
Hunk #4 succeeded at 273 (offset 1 line).
Hunk #5 succeeded at 339 (offset 1 line).
Hunk #6 succeeded at 445 (offset 1 line).
Hunk #7 succeeded at 1057 (offset 1 line).
Hunk #8 succeeded at 1080 (offset 1 line).
Hunk #9 succeeded at 2190 (offset -13 lines).
patching file src/backend/parser/analyze.c
Hunk #1 succeeded at 75 (offset -1 lines).
Hunk #2 succeeded at 1600 (offset -61 lines).
Hunk #3 succeeded at 1882 (offset -69 lines).
Hunk #4 succeeded at 1892 (offset -69 lines).
Hunk #5 succeeded at 1994 (offset -69 lines).
patching file src/backend/parser/gram.y
Hunk #1 succeeded at 158 (offset -8 lines).
Hunk #2 FAILED at 394.
Hunk #3 succeeded at 573 with fuzz 2 (offset -41 lines).
Hunk #4 succeeded at 3328 (offset -251 lines).
Hunk #5 succeeded at 10182 (offset -699 lines).
Hunk #6 succeeded at 13470 (offset -771 lines).
Hunk #7 succeeded at 13784 (offset -773 lines).
Hunk #8 succeeded at 14581 (offset -811 lines).
Hunk #9 succeeded at 14589 (offset -811 lines).
1 out of 9 hunks FAILED -- saving rejects to file
src/backend/parser/gram.y.rej
patching file src/backend/parser/parse_type.c
Hunk #1 succeeded at 736 (offset 1 line).
patching file src/backend/utils/adt/ruleutils.c
Hunk #1 succeeded at 5166 (offset -276 lines).
patching file src/include/nodes/parsenodes.h
Hunk #1 succeeded at 1285 (offset -175 lines).
Hunk #2 succeeded at 1321 (offset -175 lines).
Hunk #3 succeeded at 1350 (offset -175 lines).
patching file src/include/nodes/value.h
patching file src/include/parser/kwlist.h
Hunk #1 succeeded at 95 (offset -2 lines).
patching file src/test/regress/expected/create_view.out
Hunk #1 succeeded at 1571 (offset -55 lines).
patching file src/test/regress/expected/rules.out
Hunk #1 succeeded at 2260 (offset -85 lines).
patching file src/test/regress/expected/union.out
Hunk #1 succeeded at 59 with fuzz 2.
Hunk #3 succeeded at 479 (offset -1 lines).
Hunk #4 succeeded at 609 (offset -1 lines).
Hunk #5 succeeded at 684 (offset -1 lines).
Hunk #6 succeeded at 785 with fuzz 1 (offset -1 lines).
Hunk #7 succeeded at 838 (offset -1 lines).
patching file src/test/regress/sql/create_view.sql
Hunk #1 succeeded at 524 (offset -27 lines).
patching file src/test/regress/sql/union.sql
Hunk #1 succeeded at 20 with fuzz 2.
Hunk #2 succeeded at 69 with fuzz 2.
Hunk #3 succeeded at 149 (offset -1 lines).
Hunk #4 succeeded at 194 (offset -1 lines).
Hunk #5 succeeded at 218 (offset -1 lines).
Hunk #6 succeeded at 252 with fuzz 2 (offset -1 lines).
Hunk #7 succeeded at 281 (offset -1 lines).
..../postgresql-9.6.2$

Regards,
Zeray

On Wed, Mar 29, 2017 at 5:53 AM, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp
> wrote:

> Hi,
>
> On 2017/03/28 15:40, Kang Yuzhe wrote:
> > Thanks Tsunakawa for such an informative reply.
> >
> > Almost all of the docs related to the internals of PG are of introductory
> > concepts only.
> > There is even more useful PG internals site entitled "The Internals of
> > PostgreSQL" in http://www.interdb.jp/pg/ translation of the Japanese PG
> > Internals.
> >
> > The query processing framework that is described in the manual as you
> > mentioned is of informative and introductory nature.
> > In theory, the query processing framework described in the manual is
> > understandable.
> >
> > Unfortunate, it is another story to understand how query processing
> > framework in PG codebase really works.
> > It has become a difficult task for me to walk through the PG source code
> > for example how SELECT/INSERT/TRUNCATE in the the different modules under
> > "src/..". really works.
> >
> > I wish there were Hands-On with PostgreSQL Internals like
> > https://bkmjournal.wordpress.com/2017/01/22/hands-on-with-
> postgresql-internals/
> > for more complex PG features.
> >
> > For example, MERGE SQL standard is not supported yet by PG. I wish there
> > were Hands-On with PostgreSQL Internals for MERGE/UPSERT. How it is
> > implemented in parser/executor/storage etc. modules with detailed
> > explanation for each code and debugging and other important concepts
> > related to system programming.
>
> I am not sure if I can show you that one place where you could learn all
> of that, but many people who started with PostgreSQL development at some
> point started by exploring the source code itself (either for learning or
> to write a feature patch), articles on PostgreSQL wiki, and many related
> presentations accessible using the Internet. I liked the following among
> many others:
>
> Introduction to Hacking PostgreSQL:
> http://www.neilconway.org/talks/hacking/
>
> Inside the PostgreSQL Query Optimizer:
> http://www.neilconway.org/talks/optimizer/optimizer.pdf
>
> Postgres Internals Presentations:
> http://momjian.us/main/presentations/internals.html
>
> Thanks,
> Amit
>
>
>