Skip site navigation (1) Skip section navigation (2)

Re: Patch for handling long null terminated strings in JDBC driver

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Barry Lind <barry(at)xythos(dot)com>
Cc: "pgsql-patches(at)postgresql(dot)org" <pgsql-patches(at)postgresql(dot)org>, pgsql-jdbc(at)postgresql(dot)org, swampler(at)noao(dot)edu
Subject: Re: Patch for handling long null terminated strings in JDBC driver
Date: 2001-07-13 02:22:53
Message-ID: 200107130222.f6D2MrT27491@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-jdbcpgsql-patches
Your patch has been added to the PostgreSQL unapplied patches list at:

	http://candle.pha.pa.us/cgi-bin/pgpatches

I will try to apply it within the next 48 hours.

> The attached patch fixes problems with the JDBC driver handling long 
> null terminated strings.  The FE/BE protocol sends in some cases null 
> terminated strings to the client.  The docs for the FE/BE protocol state 
> that there is no limit on the size of a null terminated string sent to 
> the client and a client should be coded using an expanding buffer to 
> deal with large strings.  The old code did not do this and gave an error 
> if a null terminated string was greater than either 4 or 8K.  It appears 
> that with the advent of TOAST very long SQL statements are becoming more 
> common, and apparently some error messages from the backend include the 
> SQL statement thus easily exceeding the 8K limit in the old code.
> 
> In fixing I also cleaned up some calls in the JDBC fastpath code that 
> were not doing character set conversion under multibyte, and removed 
> some methods that were no longer needed.  I also removed a potential 
> threading problem with a shared variable that was being used in 
> Connection.java.
> 
> Thanks to Steve Wampler for discovering the problem and sending the 
> initial diffs that were the basis of this patch.
> 
> thanks,
> --Barry

> *** ./interfaces/jdbc/org/postgresql/Connection.java.orig	Thu Jul 12 13:37:28 2001
> --- ./interfaces/jdbc/org/postgresql/Connection.java	Thu Jul 12 13:32:55 2001
> ***************
> *** 82,92 ****
>       public int pid;
>       public int ckey;
>   
> -     // This receive_sbuf should be used by the different methods
> -     // that call pg_stream.ReceiveString() in this Connection, so
> -     // so we avoid uneccesary new allocations.
> -     byte receive_sbuf[] = new byte[8192];
> - 
>       /**
>        * This is called by Class.forName() from within org.postgresql.Driver
>        */
> --- 82,87 ----
> ***************
> *** 167,174 ****
>   		// The most common one to be thrown here is:
>   		// "User authentication failed"
>   		//
> ! 		throw new SQLException(pg_stream.ReceiveString
> !                                        (receive_sbuf, 4096, getEncoding()));
>   
>   	      case 'R':
>   		// Get the type of request
> --- 162,168 ----
>                   // The most common one to be thrown here is:
>                   // "User authentication failed"
>                   //
> !                 throw new SQLException(pg_stream.ReceiveString(getEncoding()));
>   
>                 case 'R':
>                   // Get the type of request
> ***************
> *** 238,245 ****
>             break;
>   	case 'E':
>   	case 'N':
> !            throw new SQLException(pg_stream.ReceiveString
> !                                   (receive_sbuf, 4096, getEncoding()));
>           default:
>             throw new PSQLException("postgresql.con.setup");
>         }
> --- 232,238 ----
>             break;
>           case 'E':
>           case 'N':
> !            throw new SQLException(pg_stream.ReceiveString(getEncoding()));
>           default:
>             throw new PSQLException("postgresql.con.setup");
>         }
> ***************
> *** 251,257 ****
>   	   break;
>   	case 'E':
>   	case 'N':
> !            throw new SQLException(pg_stream.ReceiveString(receive_sbuf, 4096, getEncoding()));
>           default:
>             throw new PSQLException("postgresql.con.setup");
>         }
> --- 244,250 ----
>              break;
>           case 'E':
>           case 'N':
> !            throw new SQLException(pg_stream.ReceiveString(getEncoding()));
>           default:
>             throw new PSQLException("postgresql.con.setup");
>         }
> ***************
> *** 491,497 ****
>   			{
>   			case 'A':	// Asynchronous Notify
>   			    pid = pg_stream.ReceiveInteger(4);
> ! 			    msg = pg_stream.ReceiveString(receive_sbuf,8192,getEncoding());
>   			    break;
>   			case 'B':	// Binary Data Transfer
>   			    if (fields == null)
> --- 484,490 ----
>                           {
>                           case 'A':	// Asynchronous Notify
>                               pid = pg_stream.ReceiveInteger(4);
> !                             msg = pg_stream.ReceiveString(getEncoding());
>                               break;
>                           case 'B':	// Binary Data Transfer
>                               if (fields == null)
> ***************
> *** 502,508 ****
>   				tuples.addElement(tup);
>   			    break;
>   			case 'C':	// Command Status
> ! 			    recv_status = pg_stream.ReceiveString(receive_sbuf,8192,getEncoding());
>   
>   				// Now handle the update count correctly.
>   				if(recv_status.startsWith("INSERT") || recv_status.startsWith("UPDATE") || recv_status.startsWith("DELETE") || recv_status.startsWith("MOVE")) {
> --- 495,501 ----
>                                   tuples.addElement(tup);
>                               break;
>                           case 'C':	// Command Status
> !                             recv_status = pg_stream.ReceiveString(getEncoding());
>   
>                                   // Now handle the update count correctly.
>                                   if(recv_status.startsWith("INSERT") || recv_status.startsWith("UPDATE") || recv_status.startsWith("DELETE") || recv_status.startsWith("MOVE")) {
> ***************
> *** 544,550 ****
>   				tuples.addElement(tup);
>   			    break;
>   			case 'E':	// Error Message
> ! 			    msg = pg_stream.ReceiveString(receive_sbuf,4096,getEncoding());
>   			    final_error = new SQLException(msg);
>   			    hfr = true;
>   			    break;
> --- 537,543 ----
>                                   tuples.addElement(tup);
>                               break;
>                           case 'E':	// Error Message
> !                             msg = pg_stream.ReceiveString(getEncoding());
>                               final_error = new SQLException(msg);
>                               hfr = true;
>                               break;
> ***************
> *** 559,568 ****
>   				hfr = true;
>   			    break;
>   			case 'N':	// Error Notification
> ! 			    addWarning(pg_stream.ReceiveString(receive_sbuf,4096,getEncoding()));
>   			    break;
>   			case 'P':	// Portal Name
> ! 			    String pname = pg_stream.ReceiveString(receive_sbuf,8192,getEncoding());
>   			    break;
>   			case 'T':	// MetaData Field Description
>   			    if (fields != null)
> --- 552,561 ----
>                                   hfr = true;
>                               break;
>                           case 'N':	// Error Notification
> !                             addWarning(pg_stream.ReceiveString(getEncoding()));
>                               break;
>                           case 'P':	// Portal Name
> !                             String pname = pg_stream.ReceiveString(getEncoding());
>                               break;
>                           case 'T':	// MetaData Field Description
>                               if (fields != null)
> ***************
> *** 595,601 ****
>   
>   	for (i = 0 ; i < nf ; ++i)
>   	    {
> ! 		String typname = pg_stream.ReceiveString(receive_sbuf,8192,getEncoding());
>   		int typid = pg_stream.ReceiveIntegerR(4);
>   		int typlen = pg_stream.ReceiveIntegerR(2);
>   		int typmod = pg_stream.ReceiveIntegerR(4);
> --- 588,594 ----
>   
>           for (i = 0 ; i < nf ; ++i)
>               {
> !                 String typname = pg_stream.ReceiveString(getEncoding());
>                   int typid = pg_stream.ReceiveIntegerR(4);
>                   int typlen = pg_stream.ReceiveIntegerR(2);
>                   int typmod = pg_stream.ReceiveIntegerR(4);
> *** ./interfaces/jdbc/org/postgresql/fastpath/Fastpath.java.orig	Thu Jul 12 13:37:28 2001
> --- ./interfaces/jdbc/org/postgresql/fastpath/Fastpath.java	Thu Jul 12 13:31:59 2001
> ***************
> *** 89,95 ****
>       //DriverManager.println("ReceiveChar() = "+in+" '"+((char)in)+"'");
>       //if(in!='V') {
>       //if(in=='E')
> !     //throw new SQLException(stream.ReceiveString(4096));
>       //throw new SQLException("Fastpath: expected 'V' from backend, got "+((char)in));
>       //}
>       
> --- 89,95 ----
>       //DriverManager.println("ReceiveChar() = "+in+" '"+((char)in)+"'");
>       //if(in!='V') {
>       //if(in=='E')
> !     //throw new SQLException(stream.ReceiveString(conn.getEncoding()));
>       //throw new SQLException("Fastpath: expected 'V' from backend, got "+((char)in));
>       //}
>   
> ***************
> *** 123,134 ****
>   	  //------------------------------
>   	  // Error message returned
>   	case 'E':
> ! 	  throw new PSQLException("postgresql.fp.error",stream.ReceiveString(4096));
>   	  
>   	  //------------------------------
>   	  // Notice from backend
>   	case 'N':
> ! 	  conn.addWarning(stream.ReceiveString(4096));
>   	  break;
>   	  
>   	  //------------------------------
> --- 123,134 ----
>             //------------------------------
>             // Error message returned
>           case 'E':
> !           throw new PSQLException("postgresql.fp.error",stream.ReceiveString(conn.getEncoding()));
>   
>             //------------------------------
>             // Notice from backend
>           case 'N':
> !           conn.addWarning(stream.ReceiveString(conn.getEncoding()));
>             break;
>   
>             //------------------------------
> *** ./interfaces/jdbc/org/postgresql/PG_Stream.java.orig	Thu Jul 12 13:37:28 2001
> --- ./interfaces/jdbc/org/postgresql/PG_Stream.java	Thu Jul 12 13:54:03 2001
> ***************
> *** 23,28 ****
> --- 23,29 ----
>     private Socket connection;
>     private InputStream pg_input;
>     private BufferedOutputStream pg_output;
> +   private byte[] byte_buf = new byte[8*1024];
>   
>       BytePoolDim1 bytePoolDim1 = new BytePoolDim1();
>       BytePoolDim2 bytePoolDim2 = new BytePoolDim2();
> ***************
> *** 200,271 ****
>     }
>   
>     /**
> !    * Receives a null-terminated string from the backend.  Maximum of
> !    * maxsiz bytes - if we don't see a null, then we assume something
> !    * has gone wrong.
>      *
> -    * @param maxsiz maximum length of string
> -    * @return string from back end
> -    * @exception SQLException if an I/O error occurs
> -    */
> -   public String ReceiveString(int maxsiz) throws SQLException
> -   {
> -     byte[] rst = bytePoolDim1.allocByte(maxsiz);
> -     return ReceiveString(rst, maxsiz, null);
> -   }
> - 
> -   /**
> -    * Receives a null-terminated string from the backend.  Maximum of
> -    * maxsiz bytes - if we don't see a null, then we assume something
> -    * has gone wrong.
> -    *
> -    * @param maxsiz maximum length of string
>      * @param encoding the charset encoding to use.
> -    * @param maxsiz maximum length of string in bytes
>      * @return string from back end
> !    * @exception SQLException if an I/O error occurs
>      */
> !   public String ReceiveString(int maxsiz, String encoding) throws SQLException
> !   {
> !     byte[] rst = bytePoolDim1.allocByte(maxsiz);
> !     return ReceiveString(rst, maxsiz, encoding);
> !   }
> ! 
> !   /**
> !    * Receives a null-terminated string from the backend.  Maximum of
> !    * maxsiz bytes - if we don't see a null, then we assume something
> !    * has gone wrong.
> !    *
> !    * @param rst byte array to read the String into. rst.length must
> !    *        equal to or greater than maxsize.
> !    * @param maxsiz maximum length of string in bytes
> !    * @param encoding the charset encoding to use.
> !    * @return string from back end
> !    * @exception SQLException if an I/O error occurs
> !    */
> !   public String ReceiveString(byte rst[], int maxsiz, String encoding)
>         throws SQLException
>     {
>       int s = 0;
> ! 
> !     try
> !       {
> ! 	while (s < maxsiz)
> ! 	  {
>   	    int c = pg_input.read();
>   	    if (c < 0)
>   	      throw new PSQLException("postgresql.stream.eof");
>    	    else if (c == 0) {
>    		rst[s] = 0;
>    		break;
> !  	    } else
>   	      rst[s++] = (byte)c;
>   	  }
> ! 	if (s >= maxsiz)
> ! 	  throw new PSQLException("postgresql.stream.toomuch");
>         } catch (IOException e) {
>   	throw new PSQLException("postgresql.stream.ioerror",e);
>         }
>         String v = null;
>         if (encoding == null)
>             v = new String(rst, 0, s);
> --- 201,245 ----
>     }
>   
>     /**
> !    * Receives a null-terminated string from the backend.  If we don't see a
> !    * null, then we assume something has gone wrong.
>      *
>      * @param encoding the charset encoding to use.
>      * @return string from back end
> !    * @exception SQLException if an I/O error occurs, or end of file
>      */
> !   public String ReceiveString(String encoding)
>         throws SQLException
>     {
>       int s = 0;
> !     byte[] rst = byte_buf;
> !     try {
> !       int buflen = rst.length;
> !       boolean done = false;
> !       while (!done) {
> !         while (s < buflen) {
>             int c = pg_input.read();
>             if (c < 0)
>               throw new PSQLException("postgresql.stream.eof");
>             else if (c == 0) {
>               rst[s] = 0;
> +             done = true;
>               break;
> !           } else {
>                 rst[s++] = (byte)c;
>             }
> !           if (s >= buflen) { // Grow the buffer
> !             buflen = (int)(buflen*2); // 100% bigger
> !             byte[] newrst = new byte[buflen];
> !             System.arraycopy(rst, 0, newrst, 0, s);
> !             rst = newrst;
> !           }
> !         }
> !       }
>       } catch (IOException e) {
>         throw new PSQLException("postgresql.stream.ioerror",e);
>       }
> + 
>       String v = null;
>       if (encoding == null)
>         v = new String(rst, 0, s);

> 
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026

In response to

pgsql-patches by date

Next:From: Mark VolpeDate: 2001-07-13 13:23:57
Subject: Re: [HACKERS] [PATCH] Re: Setuid functions
Previous:From: Bruce MomjianDate: 2001-07-13 00:56:55
Subject: Re: Patch to add support for partial indices

pgsql-jdbc by date

Next:From: NamrataDate: 2001-07-13 05:33:49
Subject: problen in starting the naming service using the JDBC driver!
Previous:From: Guy FraserDate: 2001-07-12 23:36:51
Subject: Re: vacuum and 24/7 uptime

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group